News for hackers. Stuff that matters.
Re:Where can I get a data dump of all HN comments? by ivan_ah on Monday February 03, 2014 @01:53AM (#7169227)
Surely there's a better way than hitting the HN servers O(7169040) times...

Does anyone have an old scrape they can post somewhere or as a torrent? Note, for my purposes bag-of-words (i.e. word counts) would be enough.

Re:Where can I get a data dump of all HN comments? by tptacek on Monday February 03, 2014 @08:08PM (#7174800)
I think you just suggested that scraping HN was a constant time operation.
Re:Where can I get a data dump of all HN comments? by rahimnathwani on Monday February 03, 2014 @02:10AM (#7169266)
bag-of-words (i.e. word counts) would be enough

Are you doing sentiment analysis?

Re:Where can I get a data dump of all HN comments? by ivan_ah on Tuesday February 11, 2014 @12:42AM (#7216009)
Sorry for the delay in replying: I didn't see your comment.

I'm working on topic modelling (Latent Dirichlet Allocation in particular). It's for a business, but I also want to solve one of my problems: reading HN addiction ;)