Re:Costs of a PostgreSQL connection
by yummyfajitas on Wednesday February 19, 2014 @10:50PM
(#7268725)
Just curious, how does that happen? I've never found myself in a situation where PGBouncer can't handle the load but Postgres can.
I'm guessing it's a situation where you are running a large volume of very easy queries, for example primary key lookups with no joins?
We do have this problem! It is only a matter of how small the PGBouncer machine is compared to the PostgreSQL. In our case the PGBouncer had to deal with 16000 req/s, coming from 700 clients going through 100 connections to the PostgreSQL.
We now have 4 PGBouncers for this DB, and we're 'load balancing' them with DNS.
I've done this, long-term, in production. This is not how PGbouncer is designed to be used, and it causes serious problems under load. (I am still doing this, but I have it on my todo list to remove this glitch from my architecture, as it caused my entire site infrastructure to fail once during a launch.) Put simply: PGbouncer is single-threaded; it is not difficult to end up in situations where the one core PGbouncer is running on to be at 100% while the remaining cores (in my case, seven) running PostgreSQL are having no issue with the traffic.