News for hackers. Stuff that matters.
Re:Perks – Effectively compute quantiles for unbounded data ... by azmenthe on Monday April 15, 2013 @06:40PM (#5554934)
This is a really awesome problem! I tackled this for my work at TempoDB and ended up going with the Q-Digest algorithm although I took a good look at CKMS. Really cool to see this implements merging streams, I remember reading that CKMS was more difficult to merge streams than Q-Digest.

If anyone is interested this was my write up for algorithm selection: http://blog.tempo-db.com/post/42318820124/estimating-percent...

Re:Perks – Effectively compute quantiles for unbounded data ... by bmizerany on Monday April 15, 2013 @06:47PM (#5554988)
Great post.

Distributing the computation was quite easy. I emailed the authors of the paper and they gave me a quick answer, which is what I implemented and tested.

I'm interested in adding more implementations of the problem to perks, like Q-Digest, along with other streaming data problems.

Re:Perks – Effectively compute quantiles for unbounded data ... by azmenthe on Tuesday April 16, 2013 @02:02PM (#5559714)
Thanks!

I use this library for Q-Digest, it's worth taking a look at for an implementation reference. https://github.com/clearspring/stream-lib