Re:Robots.txt is a suicide note
by Joakal on Tuesday May 10, 2011 @02:16AM
(#2531580)
Sounds like risking the consequences of blackhat SEO with little advantage; give one version to a typical search crawler, the other version to non-official crawler. The search engine assumes it's a bait-and-switch and automatically blacklists the website.
[ Parent
| on HN ]