Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Ask HN: Why is Facebook indexed in search engine, against robots.txt rules?. Parent comment
I tested this with: wget --user-agent "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" http://www.facebook.com/robots.txt
and diffed, but the results are the same.
I tested this with: wget --user-agent "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" http://www.facebook.com/robots.txt
and diffed, but the results are the same.