Check Your Meta Tags example: http://www.host.com or http://www.host.com/index.html URL Spider Robots GoogleBot Fast-WebCrawler Mercator Scooter IA Archiver (optional) User Agent * (optional)
