TLS is used for other protocols, e.g., SMTPS (SMTP + TLS). But there's an extra DNS query for this case, and I don't think TLS setup time is a significant cause of delays. So I don't know how useful this is.

I think most SMTP use STARTTLS with preetablished TCP connection..

There's a big push to use implicit TLS with SMTP, instead of STARTTLS. Here's a post about that:

https://blog.apnic.net/2021/11/18/vulnerabilities-show-why-s...

Not sure about real-world statistics, but the current IETF position is that SMTP STARTTLS for mail submission (not transport) is to be phased out in favour of “implicit” SMTP-over-TLS with no cleartext portion, due in part to the former being an implementation minefield[1].

[1] https://datatracker.ietf.org/doc/html/rfc8314#appendix-A

Recent submission https://news.ycombinator.com/item?id=34736416

The latency hit of the extra trip will definitely be felt by end users if the endpoint is far enough away (e.g. ~100-200ms.) You can mitigate the initial setup other ways though, like the CDN approach: terminate TLS much closer with a proxy and use a warm, pre-established backhaul connection to the origin.

Less round trips are always good, though, without any extra stuff to put in place.

Having debugged stuff for someone crazy who wanted less than 50ms global latency for a private search engine I can tell you the TLS negotiation adds significant time the first time you initialize the connection

50ms is impossible. Lowerbound is 68. One-way.

emphasis on wanted. And it was quite obtainable depending on geo location. Some locations 70 was best we could do, but most of the US east coast we were able to get around 40

I've designed my own private CDN and unless you have edges near each geo, you're not getting that kind of latency. And even after TLS negotiation you have no idea what the MSS on a link along the way will be causing fragmentation and retransmits. I've made do with some shenanigans with Noise and pre shared certs and have tried to make payload sizes as small as possible.

Yes this was using a presence in every GCP and Cloudflare and AWS edge location simultainiously all hooked together with latency based routing

Hah gotcha. Hope you got paid well for that. I do it for my own pleasure, but it's a lot of work to do as the internet just isn't designed for low latency interconnect.

Doesn't it require you to already have connected to the server once?

There's a couple of "previously connected" bits with QUIC:

- The very first connection to a site is usually HTTP/2, which requires an additional RTT compared to QUIC, as the browser doesn't yet know if the server supports QUIC. In the response, the server can advertise the presence of QUIC support with the Alt-Svc header. This support flag can also be present in a HTTPS DNS record for the domain but that isn't queried by all browsers yet. Future connections to the same server will default to QUIC once the browser is aware of support, saving an additional RTT.

- Once connected to a QUIC server, the encryption keys can be cached for future connections, allowing the client to send data with the initial connection request (so called 0-RTT). This is only safe for idempotent requests though as this part of the protocol could be replayed by an attacker.

You're right that HTTP/3 requires Alt-Svc at the moment. QUIC itself doesn't require a pre-established connection (1-RTT), which is notable for non-HTTP/3 protocols and WebTransport.

QUIC is horribly inefficient on the server side, so there are legitimate reasons to use TLS 1.3 over TCP in high traffic scenarios (like CDN servers).

Interesting, can you explain in more detail on what makes QUIC more inefficient than TLS over TCP on the server side?

Kernels and hardware have been optimized for TCP. QUIC will catch up eventually.

AFAIK, kTLS and hardware TLS offload don't solve the latency problem anyways. Those only handle the AEAD and record encapsulation/decapsulation in the critical path, where maximizing throughput is the concern. Control messages are not handled, so session establishment with client hello, cipher exchange, key exchange etc. is still done entirely in userspace, and the handshaking process is where the latency issues arise.

Oh yeah, QUIC is an improvement in terms of latency and even throughput over TCP/TLS. The claim that "QUIC is horribly inefficient" centers around CPU utilization and the cost of delivering each byte. That's where hardware offload shines but it doesn't exist for QUIC yet.

The only reason QUIC delivers throughput improvements over TCP is because it uses the same congestion control as BBR, so occasional packet loss doesn't kneecap the connection. If you use BBR TCP (or RACK TCP) on FreeBSD, you'll see the same improvement in throughput vs older TCP congestion control.

The same server that will do 375Gb/s at close to 50% idle will maybe deliver 70-90Gb/s of QUIC with the CPU maxed.

TLS+TCP delivers that performance with a lot of optimizations that just don't exist for QUIC:

- inline TLS offload (Mellanox CX6DX)

- async sendfile -- note, the above 2 mean that the kernel never even maps data from a file being sent to a client into memory, much less copies it to/from userspace like with QUIC. This cuts memory bandwidth to roughly 1/4 of what it would be with a traditional read/encrypt/write to socket server.

- TCP segmentation offload

- TCP & IP checksum offload

- TCP large receive offload

Some of these optimizations are slowly becoming available, but until all are present, QUIC will cost 2x - 4x as much to serve as TCP for a CDN workload.