Show HN: Hacker News 2018, Year in Review

Re:Show HN: Hacker News 2018, Year in Review by minimaxir on Friday January 18, 2019 @02:01PM (#18941871)

> It would be cool if we could use the public Big Query data set in Google cloud, but it is missing some data that we'd like to report on. Namely users as well as enhancing the posts with fields like (domain name of a story).

The BigQuery dataset has both of the fields necessary to do the analysis, so I'm confused. (you do need to use the NET.HOST function to extract a HN-esque domain name from the url; more info in this notebook: https://www.kaggle.com/minimaxir/hacker-news-submission-scor...)

[ Parent | on HN ]

Re:Show HN: Hacker News 2018, Year in Review by harrisreynolds on Friday January 18, 2019 @03:15PM (#18942608)

Thanks for the link! Will take a look. Would love to dump our Redshift instance!

[ Parent | on HN ]

Re:Show HN: Hacker News 2018, Year in Review by sbr464 on Friday January 18, 2019 @03:19PM (#18942645)

I'm curious, what's the current size of your instance (disk space/# records) for this dataset?

I'd be interested in a dump of the raw data, to save having to access via the API. I'd be willing to pay for the bandwidth/temp charges if any.

[ Parent | on HN ]

Re:Show HN: Hacker News 2018, Year in Review by scapecast on Friday January 18, 2019 @03:20PM (#18942655)

What is it that makes you want to dump Redshift? Sounds like there's a specific frustration?

[ Parent | on HN ]

Re:Show HN: Hacker News 2018, Year in Review by harrisreynolds on Friday January 18, 2019 @03:35PM (#18942786)

I actually like Redshift fine. But we are paying about $250/month for it just to host this data and if it was already in a publicly available database it would be nice to drop that.

But using Redshift for this has been great for us in terms of testing our tool.

[ Parent | on HN ]