Re:Trolling the search engines
by Buetol on Sunday March 16, 2014 @05:31PM
(#7411405)
Thanks, corrected it
[ Parent
| on HN ]
I had the same "fun" idea 6 years ago. In 3 months Yahoo Site Explorer reported 150 million pages indexed. As of today Google still shows 55 million pages (in comparison, https://www.google.com/search?q=site:http://en.wikipedia.org... reports 34 million pages).
I had to kill the experiment (no more new "pages" crawled) because of the CPU load and bandwidth costs, even throttling robots