News for hackers. Stuff that matters.
Re:On Counting (2017) by djhworld on Thursday May 24, 2018 @12:46PM (#17145817)
AWS Athena is pretty good at tackling this problem, or PrestoDB running on EMR.

As long as your S3 data is reasonably partitioned, and you don't have millions of small files, it does a reasonable job -even on count(distinct)

It even support approx_distinct for hyperloglog estimation too

There are no comments on this comment yet.