Every timeout in every hop of the chain is within your control to configure. Setup a subdomain and send long polling requests through that so the timeouts can be set higher and not impact regular http requests or open yourself up to slow client ddos.
Why would you try to do long polling and not configure your request chain to be able to handle them without killing idle connections? The problems you have only exist because you're allowing them to exist. Set your idle timeouts higher. Send keepalives more often. Tell your web servers to not do request buffering, etc.
All of that is extremely easy to test and verify functioanlity. Does the request live longer than your polling interval? Yes? Great you're done! No? Tune some more timeouts and log the request chain everywhere you can until you know where the problems lie. Knock them out one by one going back to the origin until you get what you want.
Long polling is easy to get right from an operations perspective.
Thank you for pointing that out. This thread alone is bound to become a meme.
This changes everything!
I better go kill myself from this embarrassment so my family doesn't have to live with my shame! Hopefully with enough generations of time passing my family will rise above the stain I've created for them!
For example, you said "Every timeout in every hop of the chain is within your control to configure", but I'm quite confident that the WiFi router my office uses doesn't respect timeouts correctly, nor does my Phone's ISP, or certain VPNs. Those "hops" in the network are not within my control at all.
There's no need to go to extremes, no matter how embarrassing and notably laughable your comment was. I'd say enjoy your fame.
Agreed, honestly if an argument is made and it makes sense, it doesn't matter who is on the other side.
> no matter how embarrassing and notably laughable your comment was.
I wouldn't even call the original comment laughable, they had a point - if you are in control of significant parts of the overall solution, then you can most likely mitigate most of the issues. And, while not a best practice, there's nothing really preventing you from sneaking in the occasional keepalive type response in the stream of events, if you deem it necessary.
The less of the infrastructure and application you control, the more likely the other issues are likely to pop up their heads. As usual, it depends on your circumstances and both the original comment and the response are valid in their own right. The "extreme" response was a bit more... I'm not sure, cringe? Oh well.
The underlying problems are those of legacy software. People long gone from Linden Lab wrote this part of Second Life. The networking system predates the widespread use of middle boxes. (It also predates the invention of conflict-free replicated data types, which would have helped some higher level consistency problems.) The internal diagnostic tools are not very helpful. The problem manifests itself as errors in the presentation of a virtual world, far from the network layer. What looked like trouble at the higher levels turned to be, at least partially, trouble at the network layer.
The developer who has to fix this wrote "This made me look into this part of the protocol design and I wish I hadn't."
More than you ever wanted to know about this: [1] That discussion involves developers of four different clients, some of which talk to two different servers.
(All this, by the way, is part of why there are very few big, seamless, high-detail metaverses. Despite all the money spent during the brief metaverse boom, nobody actually shipped a good one. There are some hard technical problems seen nowhere else. Somehow I seem to end up in areas like that.)
[1] https://community.secondlife.com/forums/topic/503010-obscure...
lol
I don't try to run red lights because I don't have control over the lights on the road.
You’ve been offered the gift of wisdom. It’d be wise on your part to pay attention, because you clearly aren’t.
How does that help? You can't pop from a queue over HTTP because when the client disconnects you don't know whether it saw your response or not.
However, queued messages don't have to be kept for a very long time, usually. Because every connection method suffers from this problem, you wouldn't usually architect a system with no resync or reset strategy in the client when reconnection takes so long that it isn't useful to stream every individual message since the last connection.
The client and/or server have a resync timeout, and the server's queue is limited to that timeout, plus margin for various delays.
Once there's a resync strategy implemented, it is often reasonable for rhe server to be able to force a resync early, so it can flush messages queues according to other criteria than a strict timeout. For example memory pressure or server restarts.
To be clear, there's no real difference - in both cases you have to keep messages in some queue and potentially resend them until they've been acknowledged.
I mean, in both cases, it's a TCP connection over (eg) port 443 that's being kept open, right? Intermediaries can't snoop the data if its SSL, so all they know is "has some data been sent recently?" Why would they kill long-polling sessions after 10sec and not web socket ones?
Why such a short timeout? Attackers can open connections and silently drop them to tie up resources. This is why we can't have nice things.
[1] https://httpd.apache.org/docs/2.4/mod/core.html#keepalivetim...
Anyways, the real takeaway is even if your current solution works now, one day someone will put something stupid between your server and the clients that will invalidate all current assumptions.
For example I have created some service which consumes a very large NDJSON file over an HTTPS connection, which I expect to be open for half an hour at least, so I can process the content as a stream.
I dread the day when I have to fight with someone’s IT to keep this possible.
100 Continue could be usable as a workaround. Would probably require at a bare minimum some extra integration code on the client side.
I think it's a premise of reliable long-polling that the server can hold on to messages across the changeover from one client request to the next.
Second Life has an HTTPS long polling channel between client and server. It's used for some data that's too bulky for the UDP connection, not too time sensitive, or needs encryption. This has caused much grief.
On the client side, the poller uses libcurl. Libcurl has timeouts. If the server has nothing to send for a while, libcurl times out. The client then makes the request again. This results in a race condition if the server wants to send something between timeout and next request. Messages get lost.
On top of that, the real server is front-ended by an Apache server. This just passes through relevant requests, blocking the endless flood of junk HTTP requests from scrapers, attacks, and search engines. Apache has a timeout, and may close a connection that's in a long poll and not doing anything.
Additional trouble can come from middle boxes and proxy servers that don't like long polling.
There are a lot of things out there that just don't like holding an HTTP connection open. Years ago, a connection idle for a minute was fine. Today, hold a connection open for ten seconds without sending any data and something is likely to disconnect it.
The end result is an unreliable message channel. It has to have sequence numbers to detect duplicates, and can lose messages. For a long time, nobody had discovered that, and there were intermittent failures that were not understood.
In the original article, the chart section labelled "loop" doesn't mention timeout handling. That's not good. If you do long polling, you probably need to send something every few seconds to keep the connection alive. Not clear what a safe number is.