> HAProxy is able to keep a connection queue and feed a maximum of (for example) 4 concurrent requests to the backend(s), thus providing back-pressure to incoming requests.

NGINX can do this as well, either with the "max_conns" parameter for upstreams, or (trickier, but perhaps more effective when the upstream is async) in combination with rate limiting:

    limit_req_zone $server_name zone=root:10m rate=100r/s;
    limit_req_status 429;

    location / {
        limit_req zone=root burst=100 delay=4;
    }