> - 6M/s ~4KiB text/plain HTTP/1.1
According to my numbers 4KiB * 6M/s is roughly 196Gbps, that really must be a very performant server, and network card.
Benchmarks are usually performed locally between a load test program and the local server. Bottlenecks are imposed at the boundaries of storage iops, memory bandwidth, cache bandwidth, cpu context switching, external io, cache miss/hits, and overhead of the program.
Anything production and performant likely leverages DPDK and multiple 40/100 Gbps nics/hcas but doesn't need to serve that much because it has CDNs and "cloudflares" in front of it. Apps with dynamic data have to be sharded in such a way that they can be horizontally-scaled. WhatsApp scaled nicely with Erlang BEAM/HiPE on FreeBSD on a low hardware budget for the volume of traffic. HiPE isn't particularly fast or efficient compared to Rust or Azul C4 JVM but it saved a lot of thread contention by making copies and saving cache invalidations by making almost everything immutable.
- 6M/s ~4KiB text/plain HTTP/1.1
- 500k/s <1KiB simple JSON responses to a single, uncached query
- 3M/s <1KiB HTTP/1.1 cached queries
If you want TLS performance, terminate it elsewhere or have a dedicated crypto (not crypto-mining) acceleration card in each server and do DSR.
Until you can afford your own 24/7 web SRE team, put Cloudflare in front of most of it and let them soak up traffic. :)
For speed, the choices usually revolve around Rust, JS+C++, or C++. Let me think: I'll take the one without buffer overflows or RCEs.
If you're trying to run a giant Spring app, good luck and take an Azul C4 GC with you on your journey.