I worked at a place whose auth server system literally serviced tens of billions of request per day. 99% of that traffic was for "dude was logged in and is looking at some page that is not unlike the one you are reading right now on HN". JWT let us cut that traffic by 99%. The remaining 1% was for servicing refresh tokens and handling requests for "sensitive" traffic (i.e. account page, anything involving writes, anything dealing with billing). This was in a system with data centers across the globe.

Auth might not be your primary bottleneck, but each page load you service has a latency budget and every millisecond spent being blocked on some auth call is a millisecond that could be spent doing else. And trust me, your database or crazy "solve it with redis" server isn't responding in 1ms, it's gonna be at least 20ms due to network overhead and the variance in latency on that request isn't always "20ms" it is "99% of the requests are 20ms and 1% are >>100ms". So now 1% of 10 billion requests (100 million requests) take way longer than 100ms... well that sucks...

Not to mention when your auth backend blows up and dies... your site is completely dead in the water. JWT can at least let you tread water for a while.

JWT is amazing. People that diss it don't understand it, or likely don't understand the tradeoffs they implicitly made with their own auth architecture.

>>> I've never worked at a company where auth or storing user sessions was a bottleneck.

I can tell you from my experience working at JP Morgan (the largest US bank) that authentication checks was the #1 root cause of major outages.

That is, except when it was #2. It was a close tie with network related issues depending on the season.

It's easy enough to understand that when sessions can't be checked, every application and service that need to check sessions are effectively 100% down.

Another hard learned fact is that (very) critical services or (very) high traffic services will simply never support remote sessions, because it's too slow and unreliable. (There are potential alternatives like JWT tokens or TLS authentication).

it's not the hash table lookup that is expensive, it's the network request.

Computers are not good at network requests because physics says they can't be - it takes orders of magnitude more time for light/electricity to traverse even one side of a datacenter to another than it does to go from the CPU side of the motherboard to the RAM side.

For plenty of use cases, the additional network roundtrip is very affordable. But it's simple math that explains why this might easily become half or more than half of the total load on your systems if you want to "just do an additional (network-mediated) hash table lookup" on every request.

The 100-2000ns it takes signals to move across a datacenter and back is not the primary cause of a network request being slow in comparison with a cpu.

LOL to 2000ns. Seriously?

10ms minimum to go to the next rack. For 99% of the requests. For the unlucky 1% you'll get outlier performance due to.... well... you spend the next sprint figuring it out and the next quarter trying to solve that scaling bottleneck.... It could be network, it could be some weird design thing, it could be redis, who knows? But yeah... enjoy figuring it out instead of focusing on your product.

1% of all requests is actually a lot of requests. And if you think 1% of your auth requests aren't 10x the median... you aren't measuring it hard enough.

What on earth kind of clownish network setup do you have where O(1) lookups over a network take 10ms? I worked on a tiny (3 people) engineering team serving 250,000 http requests/second 27/7/365 doing 5-6 over-the-network requests to other services (memcached, redis) per request we handled and our 99th percentile time to get a response on the wire was < 15 milliseconds.

"24"/7/365... we weren't warping spacetime to get responses on the wire that fast.

That's exactly my point, you are agreeing with me. I was refuting the comment claiming that networks were slow compared to on-die comms because of physics and lightspeed.

The signals going through the wire and back are 2000ns in a big datacenter. There are orders of magnitude more delays in the processing of those bits than their transmission through space via twisted pair or fiber.

It's not 10ms to go to the next rack though unless your network is broken. You're much closer to 100us (100,000ns).

Yes, it takes 2000ns for the signal to travel at the speed of light. ledauphin never talked about the time spent processing the signals, which is incredibly odd because that's where most of the time is being wasted.

This is a very common pattern in networking, people underestimate the speed of light and overestimate the speed of network equipment.