Re:Redis new data structure: the HyperLogLog
by mitchellh on Tuesday April 01, 2014 @11:51AM
(#7509179)
Yes, Redis performs well. I don't think I ever actually compared to hlld to Redis in my comment, except to say you LOSE data structures by using hlld specifically.
I was not making a comparison. I was showing another potential solution.
Re:Redis new data structure: the HyperLogLog
by antirez on Tuesday April 01, 2014 @11:55AM
(#7509208)
No problem at all with comparing! And thanks for commenting. I wrote my message since you wrote "If you're looking for much higher performance than what Redis ...". My idea is just that at the level of performance of the HLL Redis implementation to add a new component to an infrastructure for the sake of faster HLL should be considered with care, but specialized solutions may be better in many regards of course.
Re:Redis new data structure: the HyperLogLog
by VikingCoder on Tuesday April 01, 2014 @12:04PM
(#7509303)
"If you're looking for much higher performance than what Redis will get for this specific data structure"
That's a comparison, specifically claiming much higher performance.
At the same time HLLCOUNT is running in the other benchmark forcing the update of the cached cardinality as often as possible.
Yet we get around 640k ops/sec in a single core, that multiplied for the 4 cores of an Air i7 gets you to 2.4m ops/sec.
EDIT 1: Note that while I'm using random data as argument of PFADD, we are not using N HLLs but a single one, so tis benchmark is surely biased as cache misses are rare. But using multiple keys will not be terrible like losing an order of magnitude, just a reasonable percentage of OPS will go away.
EDIT 2: I had a process running in background consuming a lot of CPU, actually I tried again and reached 900k OPS/sec for core, but read EDIT 1 for more info.