How would you know if the robot is autonomous or not? They could just use a typical browser user-agent and you wouldn't know...

Hide the link from users?

Using a 1 pixel transparent GIF works - also worth putting in a warning for screen readers not to follow the link.

FWIW I've been using robots.txt honeypots on two sites for the past few years to deter scrapers. Approx 1,600 IP addresses have been banned. Most of these are for data centers.

Fuck you guy, that's pretty pricky. Is bandwidth that expensive for you?

That's uncalled for. If he owns the data and pays for the connectivity, he should be free to do as he pleases just like I can exclude you from my home or business without justification.

I like humans browsing my sites but I certainly don't want scrapers copying my work wholesale, claiming it as their own and ranking ahead of me in SERPs.

I'm not alone :-) see http://incredibill.blogspot.com/

This is what Project Honeypot does http://www.projecthoneypot.org/faq.php#f

I seem to remember there being a thread on HN about spam bot scraping for email addresses, and some of them are sophisticated enough that they can even determine if an element has been hidden through rules applied via an external CSS file.

Link a 1 pixel image and stick it in the bottle left hand corner? While someone could click on it, it is extremely unlikely.

IIRC, technically you can have a 0 pixel image in some formats (gif) though I imagine some browsers won't like that.

Sounds like risking the consequences of blackhat SEO with little advantage; give one version to a typical search crawler, the other version to non-official crawler. The search engine assumes it's a bait-and-switch and automatically blacklists the website.

That is not what zbowling has been describing.