News for hackers. Stuff that matters.
Re:Post your ideas to increase the quality of YC News by andreyf on Monday March 31, 2008 @10:21AM (#151071)
I think it's hard to talk about solutions when we don't have a solid definition of the problem. To define the problem well, I think access to the news.YC data would be very helpful.

So a meta-idea would be to open the news.YC database of users, submissions, posts, and ratings. That way, we can look for specific changes in voting/submission patterns. In particular, I'd like to know:

- how post length has changed over time

- how strong the correlation between post length and score is

- if participation has dropped among those who used to contribute the most valuable comments

- the average comment score over time

I don't see any downsides - news.YC isn't looking for profit, or to be acquired, so there doesn't see to be much use in protecting the data...

How about it, Paul?

Re:Post your ideas to increase the quality of YC News by Tichy on Monday March 31, 2008 @02:21PM (#151249)
Seems to me that data is already available, with wget?
Re:Post your ideas to increase the quality of YC News by andreyf on Tuesday April 01, 2008 @12:17AM (#151730)
Scraping seems rude, although I might try a gentle scrape if PG doesn't complain... so far, afaik, he's been silent on the topic in re to news.YC...
Re:Post your ideas to increase the quality of YC News by Tichy on Tuesday April 01, 2008 @09:39AM (#151982)
I don't think there is a robots.txt, and I remember comments that crawling HN is OK, at a reasonable rate.