Re:Where can I get a data dump of all HN comments?
by acosmism on Monday February 03, 2014 @01:16AM
(#7169137)
Scrape it
[ Parent
| on HN ]
Re:Where can I get a data dump of all HN comments?
by ivan_ah on Monday February 03, 2014 @01:53AM
(#7169227)
Surely there's a better way than hitting the HN servers O(7169040) times...
Does anyone have an old scrape they can post somewhere or as a torrent? Note, for my purposes bag-of-words (i.e. word counts) would be enough.
Re:Where can I get a data dump of all HN comments?
by tptacek on Monday February 03, 2014 @08:08PM
(#7174800)
I think you just suggested that scraping HN was a constant time operation.
Re:Where can I get a data dump of all HN comments?
by rahimnathwani on Monday February 03, 2014 @02:10AM
(#7169266)
bag-of-words (i.e. word counts) would be enough
Are you doing sentiment analysis?
Re:Where can I get a data dump of all HN comments?
by ivan_ah on Tuesday February 11, 2014 @12:42AM
(#7216009)
Sorry for the delay in replying: I didn't see your comment.
I'm working on topic modelling (Latent Dirichlet Allocation in particular). It's for a business, but I also want to solve one of my problems: reading HN addiction ;)
Re:Where can I get a data dump of all HN comments?
by GrahamsNumber on Monday February 03, 2014 @01:22AM
(#7169149)
10/10 would scrape