News for hackers. Stuff that matters.
Re:PostgreSQL 10 Released by grzm on Thursday October 05, 2017 @11:55AM (#15410173)
> PostgreSQL also used to be (is?) single-threaded, which limited performance of a single query on multi-core machines

From the submission:

"Improved Query Parallelism - Quickly conquer your analysis"

Query parallelism was introduced in 9.6 and expanded in 10.

Re:PostgreSQL 10 Released by anonetal on Thursday October 05, 2017 @12:22PM (#15410389)
Yes, I was just reading through that. The server is still single-threaded though -- they are getting the parallelism by starting multiple processes to do independent chunks of work. This makes sense for PostgreSQL, but has some fundamental limitations (e.g., it requires duplicated copies of a hash table to parallelize a hash join).
Re:PostgreSQL 10 Released by endorphone on Thursday October 05, 2017 @01:48PM (#15411024)
>The server is still single-threaded though -- they are getting the parallelism by starting multiple processes to do independent chunks of work.

So...it isn't single threaded then? I mean that is exactly how the most advanced competitors operate (Oracle, SQL Server) as well -- a given connection stays on one thread, with the advantages that confers, unless the planner decides to parallelize.

Re:PostgreSQL 10 Released by adzm on Thursday October 05, 2017 @02:59PM (#15411707)
To be technical, MSSQL uses its own bespoke scheduling, and will preempt the thread for io. All io is nonblocking. The physical thread can vary for this reason. PGSQL really does use synchronous io and a single thread though. The former is probably more scalable but the latter has been serving PGSQL fine, too.
Re:PostgreSQL 10 Released by tychver on Friday October 06, 2017 @04:21AM (#15415208)
I think bitmap heap scans have had concurrent IO for quite a while now? There's the effective_io_concurrency setting for it.
Re:PostgreSQL 10 Released by jeffdavis on Thursday October 05, 2017 @06:03PM (#15412908)
No, processes don't create fundamental limitations. They can still share memory, it's just an "opt-in" choice.

Postgres processes share memory for all kinds of things. Hash tables may be duplicated, but not due to any fundamental limitations.

Re:PostgreSQL 10 Released by halayli on Thursday October 05, 2017 @01:05PM (#15410717)
PostgreSQL uses shared memory, it doesn't copy the hash table.
Re:PostgreSQL 10 Released by anarazel on Thursday October 05, 2017 @01:52PM (#15411058)
In the specific case of hashjoins, it does build them independently right now. There's a patch to rectify that though, by putting the hashtable also into shared memory. The coordination necessary to make multi phase batch joins and other such funny bits work, unfortunately made it infeasible to get into 10.
Re:PostgreSQL 10 Released by halayli on Thursday October 05, 2017 @04:42PM (#15412436)
I stand corrected, it definitely reconstructs the hash table in each process.
Re:PostgreSQL 10 Released by anarazel on Thursday October 05, 2017 @05:16PM (#15412654)
FWIW, here's the patchset to fix that: https://commitfest.postgresql.org/15/871/
Re:PostgreSQL 10 Released by halayli on Thursday October 05, 2017 @07:35PM (#15413390)
Thanks! I remember reading this thread a while back and I thought it made it in.