View Counting at Reddit

Re:View Counting at Reddit by lucasschm on Saturday May 27, 2017 @04:33PM (#14431824)

You are correct, but HyperLogLog has many buckets counting the longest run of zeros in order to avoid the problem of outliers. I recently studied these probabilistic algorithms and did a notebook with code and plots to show their performance: https://github.com/lucasschmidtc/Probabilistic-Algorithms/bl...

[ Parent | on HN ]

Re:View Counting at Reddit by snowcrshd on Saturday May 27, 2017 @06:03PM (#14432158)

Thanks for sharing that!

Just skimmed through it and seems pretty interesting. I'll read it more in depth later.

[ Parent | on HN ]

Re:View Counting at Reddit by lucasschm on Saturday May 27, 2017 @06:48PM (#14432277)

No problem. If there are mistakes or a segment is not clear, let me know

[ Parent | on HN ]

Re:View Counting at Reddit by meetapoorvgupta on Sunday May 28, 2017 @05:37AM (#14433697)

Thanks for the write up, Lucas. It was very intuitive and I learnt a lot.

I noticed that you used 5000 buckets to store the frequency of 7000 non-unique words in the section on 'Counting Bloom Filters'. How is that better than using 7000 buckets and a uniformly distributed hash function, which would maintain frequencies perfectly? We would be using fewer buckets by an order of magnitude in a real-world implementation to save memory.

[ Parent | on HN ]

Re:View Counting at Reddit by lucasschm on Sunday May 28, 2017 @05:50PM (#14436228)

Yeah, I should have given more thought to that number. Updated the example for N=300. Thanks

[ Parent | on HN ]

Re:View Counting at Reddit by anhldbk on Saturday May 27, 2017 @07:51PM (#14432483)

Thanks for sharing guy! Interesting repo.

[ Parent | on HN ]