But this itself wouldn't cause the "struggling" you mention with more than 2 concurrent users per core

Do you know if the old open street map trick of having multiple tile servers (that are each just aliases of the same server) still works? I think this was how they tried circumventing the 6 connection limit

That's no longer necessary thanks to HTTP/2

Your webserver still has to spin off a thread for each request if you want to do substantial CPU work for each request, but rest assured you'll get all the requests at once from the browser. Not 6 at a time like in the dark ages

The RFC recommends at least 100 streams. See SETTINGS_MAX_CONCURRENT_STREAMS https://tools.ietf.org/html/rfc7540#section-6.5.2

Have you tried combining the tiles on-the-fly as image sprites?

https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Images/...