Re:Ask HN: Has Google/Bing won the search engine war?
by timr on Monday March 31, 2014 @02:34PM
(#7503264)
It wouldn't work. The crawlers that cause the problems are not the ones that pay attention to voluntary management systems like robots.txt.
[ Parent
| on HN ]
There are no comments on this comment yet.