Re:Most-mentioned links in Hacker News comments, 2006–2015
by wslh on Friday November 13, 2020 @06:39AM
(#25081268)
Does it cost money to do that or it is only time consuming?
Re:Most-mentioned links in Hacker News comments, 2006–2015
by minimaxir on Friday November 13, 2020 @10:32AM
(#25083273)
It’s not free: Only storage and egress, which are trivial.
Running the query in BQ is free up to 1 TB and the 16000 row download is free so I recommend that if necessary.
Re:Most-mentioned links in Hacker News comments, 2006–2015
by wslh on Saturday November 14, 2020 @09:08AM
(#25092362)
I think an alternative option could be to have a torrent (or other file sharing mechanism) with the public HN information. Am I missing something? 16k rows seems very tiny for doing an analysis of HN.
Re:Most-mentioned links in Hacker News comments, 2006–2015
by minimaxir on Saturday November 14, 2020 @01:26PM
(#25094342)
The vast majority of potential HN analysis can be done in SQL alone within BigQuery, which is the proper way to handle a data warehouse regardless.
You mostly need the raw data for ML model training which is a very niche use case.
If you just want a subset of the data, run a query, then save the query as a table in your project and export from there.