News for hackers. Stuff that matters.
Re:Ask PG: Polite ways to scrape HN? by benatkin on Thursday September 23, 2010 @04:40PM (#1721345)
I'm inclined to agree. Remember, pg doesn't provide an RSS feed for his essays.

http://www.paulgraham.com/rss.html

Re:Ask PG: Polite ways to scrape HN? by riobard on Thursday September 23, 2010 @06:30PM (#1721800)
Any words on why there is no API for HN?
Re:Ask PG: Polite ways to scrape HN? by benatkin on Thursday September 23, 2010 @06:47PM (#1721848)
It might be a philosophical thing. One APIs became popular, scraping started to be viewed as breaking the rules. I think this has discouraged many from writing search engines. Crawlers are just scrapers on a big scale.

I know I'd like to see more serious attempts at vertical search engines. I wonder if getting backlash, in the form of people complaining to hosting companies, would be the hardest part of writing a search engine for ruby blog posts that understands regular expressions. I could certainly use such a tool.

Re:Ask PG: Polite ways to scrape HN? by sahillavingia on Thursday September 23, 2010 @11:23PM (#1722530)
Time that could be spent working on some other, higher-marginal-utility thing, I would think.