News for hackers. Stuff that matters.
Re:Redis new data structure: the HyperLogLog by dhotson on Tuesday April 01, 2014 @09:52AM (#7508153)
So I think this is roughly what antirez is talking about with 'registers' for a counter.

From the article:

The standard error of HyperLogLog is 1.04/sqrt(m), where “m” is the number of registers used. Redis uses 16384 registers, so the standard error is 0.81%.

My understanding is that the error is very small because you're averaging over 16,000 registers.

Edit: also, I think you want the mean because it'll give you back a precise/fractional power of 2 that is pretty close to the actual count. The median would give you the nearest integer power of two, which is probably not a good estimate of the real count.

Double edit: looks like there's some research around improving averaging by throwing away outliers: http://blog.notdot.net/2012/09/Dam-Cool-Algorithms-Cardinali...

There are no comments on this comment yet.