This is precisely what happens. The matrix.org bridge has approximately 25k open client connections to freenode. It would probably be a good idea to switch it to server-server connections to save resources, but afaict nobody has experimented with that yet. If you've ever seen users with an [m] at the end of their names, that's the default name for bridged matrix users.
(Unless the matrix IRC bridge connects to the IRC server with a server-to-server type connection and can thus address messages from/to virtual users on the IRC protocol level. But that would need a lot of coordination and authorizations from IRC server administrators, which brings in so much friction that it probably kills the idea.)
edit: it seems possible that the matrix irc bridge will make a new client irc connection per bridged user, which sounds like a better idea. hard to make out from the docs though.