> I've been planning on reading the Postgres 9.6 changes for parallel queries to understand how they did the magic in a sane and controlled manner and shipped a working feature.
Very incrementally - we've been adding more and more infrastructure since PostgreSQL 9.4. Which finally was user visible with some basic parallelism in 9.6, which'll be greatly expanded in 10. There's some things that we'd have done differently if we'd started in a green field, that we had to less optimally to avoid breaking the world...
As long as you keep your eyes open to other people doing what your organization is struggling with right the first time it gives sufficient motivation to approach every problem with 'why is this here and how could we do this better.' The great thing about the state of F/OSS right now is that you have codebases that have to change because of things like large amounts of RAM being so cheap- that very well understood algorithm designed to only do things in 64MB so as to not swap out no longer makes sense and so there are intelligent motions to fix it. I've been planning on reading the Postgres 9.6 changes for parallel queries to understand how they did the magic in a sane and controlled manner and shipped a working feature.