I've always considered the #1 purpose of internal pools to be minimising latency, by moving the database connection handshakes outwith the critical path.
It's pretty common to want both, particularly for larger installations. If you have more than one or two application servers the number of connections held by the "internal" pools can become pretty large. With naive configuration you often end up with each of the internal pools being substantially over-sized, due to handling some momentary imbalance between the application servers.
In particular if your workload allows to use transaction (or even statement) level pooling, you can get very large boosts by having those two levels of pooling. The application pools ensure the connection overhead is not incurred, the external pooler ensures the total number of connections is manageable.
That's very often a reason to use external pools, yes. Or just think of a single web application that runs on multiple nodes to distribute the load on the application servers. You either carefully configure each connection pool or you use an external pooler to which each node connects to (rather than directly to the database)
Every time you add another instance, you're not adding a "happy path number" of connections. You're adding up to the internal max, because at some point you will get spammed with real or bot traffic.
It's easier to aggregate the connections than carefully balance that setup every time you make infra changes or scale up. And you need some headroom for "oh, this point release of the library spawns a thread now and gets its own connection pool... oops".
That's precisely the problem.
The usual misconfigured python/ruby application will throw 16 connections at the database.
The model for these languages is to deploy one process per core, so there will be 8 concurrent processes running. 8 cores * 16 pool = 256 connections total.
Multiply by N parallel servers and that's even more connections.
There are issues with large amount of connections and postgresql, hence people decide to put pgbouncer in front of the database to mitigate.
I am painfully aware of this due to a situation where a rogue machine with an out of date version of software kept making unwanted updates. It literally took months of off and on searching to locate the machine. And the most irritating thing was that Postgres internally contains all of the tools that we needed to trace where connections came from..which is how we knew that it was connecting through pgbouncer.
However reading docs, it looks like my complaint is dated. In version 1.6 (released in 2015) they added the application_name_add_host parameter which sticks the client ip and port to application_name. This allows bad queries to be tracked back to the source machine.
I wish I had had that feature available to me when I needed it...
I'm also curious how you got 10mb per connection (if I'm reading your parenthetical correctly). I've seen a PG box with a few hundred active connections sitting happily at just a gig of ram. Obviously WHAT you do with those connections may well change that but that isn't pg-wire overhead.
It's entirely possible to throw gigabytes of RAM as long as the company is willing to foot the bill. You're going into thousands of dollars per month depending how big and how many replicas we're talking about.
Edit: anarazel actually posted it in its reply.
This is commonly done using http://www.pgbouncer.org/
While it's not built-in, it's a really common solution and can achieve high number of connections. It comes with some issues though (no server-side prepared statements). And yes, it's not exactly the same as pipelining on the server itself since it's still limited by the number of bouncer connections, but there's usually enough idle time to sacrifice some latency for throughput.
There's also https://www.pgpool.net/mediawiki/index.php/Main_Page but I'm not familiar with the details.