Re:Most-mentioned links in Hacker News comments, 2006–2015
by wslh on Saturday November 14, 2020 @09:08AM
(#25092362)
I think an alternative option could be to have a torrent (or other file sharing mechanism) with the public HN information. Am I missing something? 16k rows seems very tiny for doing an analysis of HN.
Re:Most-mentioned links in Hacker News comments, 2006–2015
by minimaxir on Saturday November 14, 2020 @01:26PM
(#25094342)
The vast majority of potential HN analysis can be done in SQL alone within BigQuery, which is the proper way to handle a data warehouse regardless.
You mostly need the raw data for ML model training which is a very niche use case.
Running the query in BQ is free up to 1 TB and the 16000 row download is free so I recommend that if necessary.