News for hackers. Stuff that matters.
Re:The Hidden Complexity of Scaling WebSockets by arccy on Friday January 24, 2025 @05:18PM (#42817460)
I think it's more that WebSockets are held open for a long time, so if you're not careful, you can get "hot" backends with a lot of connections that you can't shift to a different instance. It can also be harder to rotate backends since you know you are disrupting a large number of active clients.
Re:The Hidden Complexity of Scaling WebSockets by dboreham on Friday January 24, 2025 @06:37PM (#42817998)
The trick to doing this efficiently is to arrange for the live session state to be available (through replication or some data bus) at the alternative back end before cut over.
Re:The Hidden Complexity of Scaling WebSockets by xyzzy123 on Saturday January 25, 2025 @07:00AM (#42821137)
When you really think about this though you start to wonder whether incrementalism is worth it, the general case is a resync from the client...

i.e, what are you buying with events streamed to client vs polling and "get me events later than I have". A robust protocol ends up looking like websockets being a way to say "something happened" but refreshes and reconnects are always running the "please catch me up on everything" case.

Re:The Hidden Complexity of Scaling WebSockets by superjan on Saturday January 25, 2025 @04:25AM (#42820551)
Assuming you control the client code, you can periodically disconnect and reconnect. This could also simplify deployment.
Re:The Hidden Complexity of Scaling WebSockets by Doxin on Tuesday January 28, 2025 @08:43AM (#42852190)
Another option is to have the client automatically reconnect. That way your backend can just drop the connection when it needs to, and the load balancing infra will make sure the reconnection ends up on a different server.

Of course you do want to make sure the client has exponential backoff and jitter when reconnecting, as to avoid thundering herd problems.

The relevant state will need to be available to all servers as well. Anything that's only known to the original server will be lost as it drops connections. On a modern deployment with a database and probably redis available this isn't too big of an ask.