News for hackers. Stuff that matters.
Re:Odyssey – Scalable PostgreSQL connection pooler by doh on Wednesday May 30, 2018 @11:39AM (#17188573)
At a certain scale the pooling becomes the bottleneck. PGB has to keep state of the connection and manage it’s life time. All this currently happens on a single core. So it doesn’t help you to tune postgres itself if PGB doesn’t keep up.

Many solve this by running multiple instances of PGB (usually each on a dedicated processor) and use some kind of load balancing (haproxy, DNS, ...) to balance the connections.

This fork removes the need for the load balancing as it does it out of the box.

BTW this is only an issue if you’ve many connections to Postgres. We have thousands servers[0] connecting in and also run a citus[1] where the queries are distributed to many workers (with addition of citus MX that now allows each server to behave as a coordinator)[2].

At a small scale you are fine with the default postgres though.

[0] https://cloud.google.com/customers/pex/

[1] https://www.citusdata.com/customers/pex

[2] https://www.citusdata.com/blog/2016/09/22/announcing-citus-m...

Re:Odyssey – Scalable PostgreSQL connection pooler by vageli on Wednesday May 30, 2018 @11:58AM (#17188781)
Interesting to see how you guys do things. My team has taken the opposite approach, deploying pgbouncer as an ECS service with multiple containers per host, fronted by an lb.
Re:Odyssey – Scalable PostgreSQL connection pooler by doh on Wednesday May 30, 2018 @12:42PM (#17189200)
We also did it this way before. However we paid for it in latency, and because we operate in cloud, network micro-outages. Once we moved it directly to the Postgres servers and deployed this fork, we saw pretty significant overall improvements.