Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Google's robots.txt. Parent comment
I added that to the last.fm/robots.txt many years ago (http://www.wired.com/business/2010/08/robot-laws/all/), and have just been made aware that it appears to have spread across the internet:
https://www.google.co.uk/search?safe=off&q="Disallow%3A+%2Fi...
Favourite sites I've found with it so far: php.net, princeton.edu, nest.com, songkick.com.
There are no comments on this comment yet.
I added that to the last.fm/robots.txt many years ago (http://www.wired.com/business/2010/08/robot-laws/all/), and have just been made aware that it appears to have spread across the internet:
https://www.google.co.uk/search?safe=off&q="Disallow%3A+%2Fi...
Favourite sites I've found with it so far: php.net, princeton.edu, nest.com, songkick.com.