Robots.txt is a suicide note

Re:Robots.txt is a suicide note by pyre on Tuesday May 10, 2011 @02:08AM (#2531561)

I seem to remember there being a thread on HN about spam bot scraping for email addresses, and some of them are sophisticated enough that they can even determine if an element has been hidden through rules applied via an external CSS file.

[ Parent | on HN ]

Re:Robots.txt is a suicide note by Aloisius on Tuesday May 10, 2011 @02:55AM (#2531657)

Link a 1 pixel image and stick it in the bottle left hand corner? While someone could click on it, it is extremely unlikely.

IIRC, technically you can have a 0 pixel image in some formats (gif) though I imagine some browsers won't like that.

[ Parent | on HN ]