Most people realize the importance of creating a search engine friendly site, but many don’t take the final step of assuring that search engine spiders or bots can fully access the site. Even worse, they fail to block bots from non-public parts of the site, or don’t recognize rogue bots that are crawling a site to steal content or for other nefarious reasons. In today’s SearchDay article, The Taming of the Bots, guest writer Tony Wright has coverage of a recent SES panel where search marketers and representatives from search engines offered tips on managing bots, whether their intent is good or ill.
When Good Search Bots go Bad
Most people realize the importance of creating a search engine friendly site, but many don’t take the final step of assuring that search engine spiders or bots can fully access the site. Even worse, they fail to block bots from non-public parts of the site, or don’t recognize rogue bots that are crawling a site […]