Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Robots.txt meant for search engines don’t work well for web archives. Parent comment
For instance, is it OK to crawl a blog with explicit copyright, save that data, then publish it elsewhere?
There are no comments on this comment yet.
For instance, is it OK to crawl a blog with explicit copyright, save that data, then publish it elsewhere?