News for hackers. Stuff that matters.
Re:Use SQL subqueries to count distinct 50x faster by jeffdavis on Friday January 24, 2014 @11:29AM (#7115707)
In postgres, distinct aggregates are a blind spot in the planner.

DISTINCT itself is handled well, e.g. "SELECT DISTINCT x, y ...".

But the distinct aggregates are not really seen by the planner, and they are always implemented with a sort. So this is a peculiar case for postgres.

One reason is because distinct aggregates can be more complicated to plan when you have a case like: "SELECT COUNT(distinct x), COUNT(distinct y) ..." because you actually need two different distinct sets as input.

Some databases have solutions for that problem, but postgres does not.

Re:Use SQL subqueries to count distinct 50x faster by aaronblohowiak on Friday January 24, 2014 @11:58AM (#7115912)
Well-documented MIT-licensed code. You can help!
Re:Use SQL subqueries to count distinct 50x faster by jeffdavis on Friday January 24, 2014 @12:16PM (#7116031)
Agreed. There's a commit fest going on right now, anyone should feel free to jump in and review a couple of patches with us. Even if you don't know C.

https://commitfest.postgresql.org/action/commitfest_view?id=...

Review is a good way to learn about new features, by the way. That's the main reason I'm able to actually write recursive queries, for instance.

Re:Use SQL subqueries to count distinct 50x faster by elliptic on Friday January 24, 2014 @01:13PM (#7116376)
Serious question - how realistic is that? How many people make significant changes to the planner/executor besides Tom Lane?
Re:Use SQL subqueries to count distinct 50x faster by jeffdavis on Friday January 24, 2014 @02:10PM (#7116877)
This particular change, in it's complete form, would be challenging in my opinion. Many other kinds of planner changes aren't so bad though, but people (including me) tend to avoid them.

Executor changes are much less intimidating and lots of people make executor changes. In my opinion, the executor is the most approachable/easiest major component.

I would encourage people to not get intimidated though. For every project where I was determined enough, I succeeded. Tom Lane does especially amazing things, of course, but I think reasoning like "I'm not Tom Lane, therefore the planner is too hard" is the wrong lesson to take away.

Re:Use SQL subqueries to count distinct 50x faster by calinet6 on Saturday January 25, 2014 @09:44AM (#7120894)
For reference, a college-level CS databases course I took had us hack on the Postgres query planner to improve a very specific case such as this.

It's large and slightly complicated, but damn it's clean and understandable. Or at least it was back in 2005...