Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Poll: Should this site be visible to Yahoo, MSN crawlers?.
If not, a simple robots.txt would fix the crawler-traffic problem without prejudicially banning any upstart search engines:
User-Agent: * Disallow: /x?
There are no comments on this comment yet.
If not, a simple robots.txt would fix the crawler-traffic problem without prejudicially banning any upstart search engines: