Making Postgres Bloom

Re:Making Postgres Bloom by matsur on Thursday July 30, 2015 @04:23PM (#9977763)

Semi-related in the land of Postgres and probabilistic data structures -- Redshift supports APPROXIMATE COUNT. Much, much faster than a raw COUNT, and their stated error is +-2%

http://docs.aws.amazon.com/redshift/latest/dg/r_COUNT.html

[ Parent | on HN ]

Re:Making Postgres Bloom by striking on Thursday July 30, 2015 @04:39PM (#9977885)

I'd eat my hat if it didn't use a Bloom filter in some way.

[ Parent | on HN ]

Re:Making Postgres Bloom by usman-m on Thursday July 30, 2015 @04:48PM (#9977966)

It probably uses a HyperLogLog--the 2% error rate kind of gives it away. Bloom filters approximate set membership queries, HyperLogLogs approximate set cardinality queries. COUNT DISTINCT is a set cardinality query.

We actually support a HyperLogLog backed COUNT DISTINCT aggregate too: http://docs.pipelinedb.com/aggregates.html#general-aggregate...

[ Parent | on HN ]

Re:Making Postgres Bloom by striking on Thursday July 30, 2015 @05:35PM (#9978325)

Consider my metaphorical hat eaten. Thanks for the cool tools! I'm currently working with Postgres and this looks like a great thing to add to the mix.

[ Parent | on HN ]

Re:Making Postgres Bloom by anarazel on Thursday July 30, 2015 @10:48PM (#9979656)

There's a postgres extension that implements hll for postgres. Rather useful: https://github.com/aggregateknowledge/postgresql-hll

[ Parent | on HN ]