Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Build a crawler to crawl million pages with only one machine in just 2 hours. Parent comment
It was just shoehorned.
The crux of a project such as this is maintaining a connection pool and managing it efficiently.
Also respecting robots.txt which the author barely mentions.
This is a "tool looking for a problem" kind of post.
There are no comments on this comment yet.
It was just shoehorned.
The crux of a project such as this is maintaining a connection pool and managing it efficiently.
Also respecting robots.txt which the author barely mentions.
This is a "tool looking for a problem" kind of post.