News for hackers. Stuff that matters.
Re:The Evolution of Data at Reddit by maslam on Wednesday March 07, 2018 @01:29PM (#16538572)
"Third party vendor". Is this Qubole? If so, would be nice to give him an Internet high-five by naming them :)
Re:The Evolution of Data at Reddit by RandComment on Wednesday March 07, 2018 @04:29PM (#16540071)
https://www.flydata.com/blog/hive-and-redshift-a-brief-compa...

Used Redshift for our large data needs. No reason to look to anything else, given our time-shifted data could easily be fit into Redshift. For larger data, we would have to have some other solution.

Re:The Evolution of Data at Reddit by danielvinson on Wednesday March 07, 2018 @02:28PM (#16539068)
This is extremely interesting and I'm eager to see the follow-up.

I'm currently planning something very similar at my company and any reference to how other smaller-scale companies handle this when scaling is invaluable. Are there any other blogs from other companies which have talked about similar things?

Re:The Evolution of Data at Reddit by dchuk on Wednesday March 07, 2018 @03:06PM (#16539353)
Not a direct answer to your question, but I maintain an HN-like aggregator of engineering blogs here: https://engineered.at

These types of posts pop up there pretty frequently

Re:The Evolution of Data at Reddit by overcast on Wednesday March 07, 2018 @04:44PM (#16540208)
These need to be sorted by publishing time, not by your database entry time. You can tell because you've recently added Percona, and it's filling up the top with backdated stuff. I'd also have preferred just a minimalist interface. We don't need anymore voting/commenting stuff at this point.
Re:The Evolution of Data at Reddit by dchuk on Wednesday March 07, 2018 @09:02PM (#16541685)
Interesting feedback. Not sure how I could make the interface more minimal, it's pretty stark as it is.

The backdated stuff is weird I'll admit, but they also posted a bunch of things in the last couple days so could be screwing with the ranking algo. I'll check it out.

Re:The Evolution of Data at Reddit by behohippy on Wednesday March 07, 2018 @02:56PM (#16539290)
Just google for <Company> engineering blog. I know for sure Facebook and Google have good ones.
Re:The Evolution of Data at Reddit by huac on Wednesday March 07, 2018 @03:11PM (#16539390)
> The tool we used did not support using Hive as a backend, and given the speed of even relatively basic Hive queries, it wouldn’t make much sense to use that for reporting metrics.

Did Reddit try setting up Presto? Analytics, not just visualization, with Hive is really hard.

Re:The Evolution of Data at Reddit by citilife on Wednesday March 07, 2018 @03:29PM (#16539559)
I've had a great experience with Hive!

One thing that has made me sad is that there is that Rails / Ruby has little to no support. Many of the repos are over four to five years old.

Re:The Evolution of Data at Reddit by mephitix on Wednesday March 07, 2018 @01:55PM (#16538792)
I think nowadays they use Microsoft PowerBI for dashboards/reporting