News for hackers. Stuff that matters.
Re:Trolling the search engines by solox3 on Sunday March 16, 2014 @05:24PM (#7411365)
> Can’t believe that: No one did it before me (to my knowledge)

This has certainly been done before: http://en.wikipedia.org/wiki/Spider_trap

Re:Trolling the search engines by scoot on Monday March 17, 2014 @11:23AM (#7415561)
The wikipedia article doesn't mention how long this has practice existed, but I know this goes back to at least the late 90s.

Circa '98 IIRC, there was a module available for a webserver I worked with which generated a page with number of bogus email addresses per page, and a number of random urls per paged that when followed generated yet another page of bogus email addresses and links.

You hid the link somewhere on a legitimate page, added the base path as an exclude in robots.txt, and any mail harvesting spam-bots would get sucked in.

The idea may well have been around longer still.

Re:Trolling the search engines by Buetol on Sunday March 16, 2014 @05:31PM (#7411405)
Thanks, corrected it
Re:Trolling the search engines by ozh on Monday March 17, 2014 @05:57AM (#7414062)
I had the same "fun" idea 6 years ago. In 3 months Yahoo Site Explorer reported 150 million pages indexed. As of today Google still shows 55 million pages (in comparison, https://www.google.com/search?q=site:http://en.wikipedia.org... reports 34 million pages).

I had to kill the experiment (no more new "pages" crawled) because of the CPU load and bandwidth costs, even throttling robots