Given, say, $50 million of dev time, what would you go about fixing? And in what way?
The RFCs are a bit elaborate so folks interested might want to look at this instead[1], which has one of the RFC authors explaining the basics of QUIC and HTTP/3.
SPDY → HTTP/2
QUIC → HTTP/3
[1]: <https://github.com/skywind3000/kcp>
[2]: <https://www.v2fly.org/en_US/>
[3]: <https://github.com/skywind3000/kcp/blob/master/README.en.md>
Does not handle unreliable messages, still only (multi)streaming, no direct support for multicast, 0-rtt which need a lot of stuff to be manually done TheRightWay or risk amplification attacks, the (imho) under-researched (and removed) forward error correction, and more.
I just restarted working on what I consider to be the solution to this, federated authentication and a bit more, but $50M is too far to be even a dream since I am not google.
Also, they have a transport (Pony express) developed specifically for RPCs, rather than byte streams or datagrams.
Links: https://research.google/pubs/pub51341/, https://research.google/pubs/pub50590/, https://research.google/pubs/pub48630/, more generally https://research.google/pubs/?area=networking
This is pure speculation based on seeing the word ASIC in one of the summaries but it seems like it could be reasonable.
They have a new system called Snap although judging from the paper I don't think it can completely replace TCP: https://research.google/pubs/pub48630/ My understanding is that Snap enables new use cases including moving functionality previously done via RPCs to RDMA-like one-sided operations. I think it is complement to RPCs but does not replace it.
This is ridiculous.
Hyperscalars see an immediate ROI from efficiency/reliability improvements and actively invest in TCP alternatives all of the time. It's just really hard.
Networking companies see an ability to differentiate their products from their peers and work on this kind of thing as well. I did a 3 second google for "QUIC acceleration Mellanox" and got a hit on Nvidia's blog right away.
You just can't trivially replace something with an investment totally 50 years of clock time and thousands of years of engineer time. It will either take a long time or a massive shift in needs/technology. FWIW, I wouldn't be surprised if the high-performance RDMA networks being put together for AI workloads were the thing that grew into the "next" thing.
It's not just the size of the investment, it's that it's the protocol everyone uses to talk to other people's machines, and you can't upgrade or replace other people's machines.
Google made that happen almost unilaterally via their Chrome dominance.
TCP gives you three things: 1. Reasonable performance - This is hard but not impossible to replicate 2. Reliability - This is very hard to replicate because networking edge cases are very hard to isolate 3. Fairness - this one is roughly impossible, because the "fairness" is an artifact of the experimentation and tweaking of Congestion Control Algorithms.
To elaborate on fairness, dynamic traffic control of all flows within a DC while maintaining high utilization is roughly impossible. You can get really close to this by picking your battles wisely (i.e. solid demand control for data warehouse workloads), but you'll always end up counting on individual flows to react appropriately to loss. They need to back off enough to make room for others without tanking their own throughput.
The people who design and implement these algorithms are definitely geniuses, but even they rely on TONS of empirical evidence to narrow parameters to what's appropriate. Of the Kernel Networking people I've worked with, Lawrence Brakmo had the most sophisticated network testing harness I've seen. Even then, you don't really know if it works (and can't finish tuning it) until you run it in production.
Running novel congestion control algorithms in production at a sufficient scale to figure out whether or not they're working appropriately is a great way to kill your network, so we end up conducting the equivalent of CCA drug testing to roll it out slowly and safely.
The end result of all of this is that it's really hard to solve the "arbitrary connections sharing arbitrary network topologies with high utilization" problem quickly enough for it ever to look like a breakthrough rather than just steady progress.
It's also worth noting that it's usually easiest to prove performance, so you'll see a lot of excitement about performance benchmarks from people who don't yet know what they're about to learn about networking. We were very much in this camp at Facebook when we were all-in on memcache-over-udp, and we later abandoned it completely.
UDP used successfully many places.
I’ve found UDP to be great for latency but pretty awful for throughout. Especially over longer routes (ie inter-region transports). Also, if you fire UDP packets out of a machine in a tight loop then there is every chance you could overload various buffers and just loose them (depending on the networking hardware).
TCP is comparatively amazing for throughput, but you do take a latency hit (especially on the initial handshake, which doesn’t exist for UDP).
There are some very experienced people commenting here though, and I’d be happy to be corrected or expanded upon.
With OpenVPN it's easy enough to test - configure for UDP, or configure for TCP. With long latency, and a tiny amount of packet losses, running TCP over TCP OpenVPN completely stalls, while TCP over UDP OpenVPN is excellent - it's around the same performance as running direct TCP, or sometimes actually better. At work we've also used other types of VPN setups (for engineers on the road), and the TCP based ones (we've used several) work fine most of the time, but if you try that from far away it becomes nearly unusable while UDP OpenVPN continues to work basically just fine.
The TCP over TCP VPN performance problem (over long latency links) presumably has to do with windowing and ack/nak on top of windowing with ack/nak.
Because the underlay TCP is lossless (being TCP), every time the overlay TCP has to retransmit, it adds to the queue of things that the underlay TCP has to retransmit (and the need to retransmit happens more or less at the same time).
So instead of linear increase in the number of packets, you get ~quadratic.
This balloons the required throughput needed to “rectify” the issue from the protocol standpoint at both levels - usually precisely at the point when there’s not enough capacity in the first place (the packet loss is supposed to signal congestion).
If you are very lucky, the link recovers fast enough that this ballooning is small enough to be absorbed by the newly available capacity.
If the outage is long enough, the rate of build-up of retransmits exceeds the capacity of the network to send them out - so it never recovers.
Needless to say, the issue is worse with large window in overlay TCP session - e.g. a sudden connectivity blip in the middle of the file transfer.
My guess is that this is in part because achieving high throughput on IP is hard and in part because it's never going to be super efficient at this level (in userspace, on top of kernel infrastructure that might not be as optimized towards throughput like it is in the case of TCP).
UDP/multicast can provide excellent throughput. It's the de facto standard for market data on all major financial exchanges. For example, the OPRA feed (which is a consolidated market data feed of all options trading) can easily burst to ~17Gbps. Typically there is a "A" feed and a "B" feed for redundancy. Now you're talking about ~34Gbps of data entering your network for this particular feed.
Also, when network engineers do stress testing with iperf we typically use UDP to avoid issues with TCP/overhead.
Depending on where your RP's are, and how you are transmitting multicast packets across a core, multicast performance can vary a lot.
The main advantage of multicast however, is that throughput between RP's doesn't need to be very large..
Market data typically comes from the same RP per exchange in most cases. Some exchanges split them by product type. Typically there’s one or two ingress points (two for redundancy) into your network at a cross connect in a data center.
That being said, it wouldn’t surprise me if they were pushing 17G of UDP on 100G transports. Probably with some pretty high-end/expensive network hardware with huge buffers. I.e you can do it if you’ve got the money, but I bet TCP would still have better raw throughput.
In general UDP is not a problem in the space because of overprovisioning. Think "algorithms are for people who don't know how to buy more RAM", but with a finicial industry budget behind it.
I wouldn’t use it to ping my gateway though, or to join a multicast group, nor would I use it to establish my bgp session, I use icmp, igmp and tcp for that.
The really key point there is probably the size of the responses, it wasn't just tiny atomic bits of stock information.
At one point as a system engineer I actually had to bump up the size of UDP socket data that the kernel would allowed to be sent across the entire production set of servers. SWEs were really hammering on UDP hard (the platform framework was sort of "sold" as being better than TCP though which doesn't have those kinds of limits).
The result was that one Christmas the traffic scaled up to the point that the switch buffers were routinely overflowing all the time. There was no slow start in UDP so the large payloads the SWEs were sending would go out as fast as the NICs could send them, which resulted in filling up packet buffers in the 6509s (Sup 720s I think at the time? Whatever it was the network engineers had already upgraded to whatever was Cisco's latest and greatest at the time and had tuned the switch buffers).
What made it even more fun was that as packets were dropped on the multicast routes the unicast replies created a bit of a bandwidth-amplification-attack. Then eventually the switch buffers started dropping IGMP packets, and if you drop enough of those in a row then IGMP sniffing fails and the multicast routes themselves start getting torn down. Now you get "packet loss" on one of the destination nodes which is complete. Then when it eventually rejoins it has fallen far behind all the peers (causing a bunch of issues when it was out of synch to begin with though) and then it requests more unicast messages to get caught up, creating even more of a flood of rapidly-sent UDP.
What I wound up doing is writing scripts to log into all the core switches and dump out the multicast tables and convert the IGMP snooped routes into static routes and reapply them. That let the multicast network grow as the site had to scale for Christmas, but kept all the routes in place and avoided the IGMP route flapping.
But even with that band-aid it still didn't work well and there was still high congestion and packet loss across the core switches. There were also problems with the CPU on the switches and Amazon drafted an extension to how multicast routing was done and got Cisco to implement it ("S,* routing" IDK if that's right its been 20 years). And it was a good job that the Network Engineers had ripped out spanning tree and gone L3 entirely since the packet loss and CPU congestion would have caused spanning tree to flap which would have amplified all the congestion issues. Eventually Tibco RVD was ripped out and a TCP-based gossip-based-clustering protocol was put into place.
So if you use UDP based stuff the datapackets need to be small, or else you need to throttle the senders somehow, and you need to not care about reliability. For stock ticker information it might work well, and for multimedia streaming where the protocol layer above it does slow start and congestion control. I suspect that if you dug up the network engineer responsible for those networks though that they could tell you stories about packet loss. If UDP works well at your company my suspicion is that you've either got a protocol sitting on top of UDP which implements at least half of what TCP offers, and/or you've got an overworked network engineer trying to keep it all together, and/or you just haven't scaled enough yet. I also wouldn't be too surprised if some wall street firms have switched to RDMA-over-Infiniband or something like that with link-layer and end-to-end based flow control (as this paper points out, though, RMDA has issues itself and doesn't meet all the criteria for a TCP-replacement, but that would at least stop the packet loss issues due to buffers overflowing).
QUIC is a good example of what you need to do in order to use UDP (Section 4 of RFC 9000 is all about Flow Control to prevent fast senders from DoS'ing your network switches). But for the average HN/reddit reader who reads something about how TCP is awful and has the "showerthought" of wondering about why everyone doesn't just switch to UDP in the datacenter, they're missing a massive problem in that Ethernet has no flow control and just promiscuously drops packets everywhere, so if you thoughtless slap UDP on top of that your datacenter will absolutely have a meltdown. You need to use something like QUIC at a minimum.
And buried in what I wrote above is an observation that UDP multicast doesn't really solve reliable delivery across multiple servers and failover of streams that you'd like to be able to see, that's another solution which is simple and wrong (and which it looks like Homa is trying to address).
Isn't this what pause frames and Pfc are for? (Honest question)
A bad client can still cause problems though, like sending a high rate of multicast packets with a TTL of 1.
Oh gotcha. It's right there in the title, but missed it somehow :p
Maybe we were just early in giving (HFT) customers RDMA back in ~2007[1][2] but I don't see it entering the mainstream anytime soon. And after a relatively short 20 years of adoption, the "next" thing for hyperscalers is not going to be the next thing for everyone else.
[1] https://downloads.openfabrics.org/Media/IB_LowLatencyForum_2...
[2] https://www.thetradenews.com/wombat-and-voltaire-break-milli...
In large web company datacenters, RDMA and RoCE have had a much "rockier" path forward.
And yet every time hardware designers get the chance they redesign Ethernet and IPv4--poorly.
See: HDMI 2.0+, USB 3.0+, Thunderbolt 3.0+, etc.
My suspicion is that this paper works fine beween pairs of peers and immediately goes straight to hell after that. It is extremely suspicious that there is zero mention of SCTP and only compares to TCP and not UDP.
The problem with RPC is that multiple organizations must agree on meaning. And that's just not going to fly. It is damn near a miracle that a huge number of institutions all agree on the Ethernet/IP command "Please take this bag of bytes closer to the machine named: <string of bytes>."
Here's an example: I connected my nice expensive audio interface to my Thunderbolt port. It worked great! Then I moved a window on my monitor and all hell broke loose. In spite of the fact that it had way more than enough bandwidth to handle everything.
See, Thunderbolt doesn't have the ability to say "This tiny packet going to there needs priority and you need to break up those giant display packets."
Ethernet has solved problems like these in standards. They're not always implemented on particular chipsets, but they exist, and you generally can buy a product that has them.
Everything Ethernet has done and standardized has generally been for a reason. If you don't implement Ethernet, then you are starting over from scratch and will have to reimplement all of that stuff.
And you're probably not smarter than the guys who did it for Ethernet.
(If I'm being charitable: what's happened is that lot of standards tried to be more cost optimized than Ethernet. The problem is that transistor prices keep coming down. Eventually the price delta between Ethernet and <whatever> becomes inconsequential and you're basically left with real Ethernet and "kinda crappy" Ethernet at almost the same price.)
What you're talking about though isn't just coming up with new ideas or even new products. It's replacing hundreds of billions in infrastructure wholesale. The scale at which these changes needs to happen to be practical are at the cluster level in a single data center. If you can propose something that fits that bill there are a few companies willing to pay you millions in salary as an engineering fellow to do it.
I'd put it differently: it's paying up hundreds of billions in infrastructure to have some sort of gain.
And which gain is that exactly?
I see a lot of "the world is dumb but I am smart" comments in this thread but I saw no one presenting any clear advantage or performance improvement claim about hypothetical replacements. I see a lot of "we need to rewrite things" comments but not a single case being made with a clear tradeoff being presented. Every single criticism of TCP/IP in this thread sounds like change for the sake of change, and changes that aren't even presented with tangible improvements in mind or a clear performance gain.
Wouldn't that explain why TCP is so prevalent, and no one in their right mind thinks of replacing it?
Are there actual numbers demonstrating this?
I mean, people are advocating wasting billions revamping infrastructure. What kind of performance are you hoping to buy with those billions? And are those gains worth it, or is just sake for the sake of change?
Sometimes things are indeed good enough.
It amuses me that many of those saying "we need to change" are the same ones that bemoan it when car manufacturers remove buttons or make glove boxes operational from a touch screen because they can.
The only way this would be safe is if you own the entire network including the computers, otherwise you're leaking info.
That said, I used to do this with Wireshark apliances when I was a network engineer, which is cheating but I could grab all the packets from a transaction and export to the Wireshark desktop app to analyze.
I also remember the Cisco reps trying to sell something that would do something closer to what you describe, but it that was almost a decade ago and a proprietary protocol anyway.
If this was true then how do you explain that the likes of AWS, the same company who ended up investing in developing their own processor line, doesn't seem to agree that none of the pet peeves you mentioned are worth fixing?
Isn't it plausible that graviton was designed yet TCP retained simply because graviton as a project is easier to complete successfully?
This is just not true. Stuff needs to be funded and worth doing, and the internet, like almost everything, is built on making things worth paying for, but there are also loads of improvements everywhere are being made.
I think this is more of an artefact of horizontal scaling and port-contention. De-facto standard discovery mechanism DNS does not work with ports, so "well-known port" abstraction kinda fails. Http as tunnel mostly avoids/sidesteps this problem.
> We should have a way to forward every single layer of the stack across each hop, and return back each layer of the stack, so that we can programmatically determine the exact causes of network issues, automatically diagnose them, and inform the user how to solve them.
This is weird take or I don't understand it. If you can communicate with an edge node in another network, but the edge node has issues communicating with some inner node (on your behalf), then, as a user, you have no hope of fixing that connectivity issue anyway, regardless of whether layered approach is used or not. This may be related to previous point about http as universal tunnel. Yes, this is a problem, but in a way that communications are effectively terminated at the edge node and monstrosity of stuff happens behind the scenes
Yes, it does, see SRV records.
This is severe bullshit on two fronts:
- there is an immediate return on value - Google was driving this a decade+ ago for improvements in the data center (things like doubled+cancelable rpc, tcp cubic, quic, etc)
- academia constantly attempts to make these improvements as well because researchers are super incentivized to dethrone tcp for the glory. There are constant attempts to re-invent various layers (IP, tcp, the non existent upper layers of the OSI, etc) that come out of academic conferences every year.
The reason we’re still here is because our current stacks have been heavily optimized and tooled for production workloads. NICs can transparently re-assemble TCP segments for the OS and they can segment before transmit. You have to have a damn good value prop to throw away everything from software and hardware to careers and curriculum. It has to be a shitload better than the security nightmare of “return back each layer of the stack”.
I work in a company that builds network troubleshooting/observability tools and we have some pretty experienced analysts to tell you what's wrong with the network. With that context, your idea of having any tool automatically diagnosing network issues is a pipe dream.
The problem with networks is that they're very complex systems, with multiple elements along the way, made by different manufacturers, often with different owners, failures aren't always easily reproducible, and with human configuration (and therefore errors) almost every step of the way. Even if a tool that "returns each layer of the stack" would be useful, it still would be far from enough to diagnose issues.
That's how bad interoperability in the early days was!
If you have ever used multiple TCP or UDP connections in parallel on a single machine (doesn't matter if server or client) then you should realize that ports are required.
Apart from that, you can run HTTP on other ports than 80. You can also use HTTP to load balance or do service discovery by means of redirects. (Caveat, I don't work in this field and can't say how solid the approach works in practice).
The basic reason is that software at every level expects TCP/IP. And you can't drop in a translation layer because it will require at least the same amount of overhead as "real" TCP/IP.
It is not a local problem, it is a global problem that affects basically every single piece of non-trivial software in existence.
Even if you construct your datacenter with the new protocol you will run into problems that you can't run anything in it. Want Python? Sorry, have to rewrite it. And every Python library. And every Python application. Then you need to deal with problems that people who can run their scripts on their machines can't run them in datacenter. And so on.
The reason nobody wants to do this is that they would be investing huge amount of money to solve a problem for everybody else. Because the only way to make TCP/IP replacement work is to make it completely free and available to everybody.
There are much better ways to allocate your funds and precious top level engineers that let them distance themselves from competition temporarily.
This violates the principle of encapsulation that the entire field of networking is based on, not to mention an massive security hole.
Thats virtual networking. but that introduces latency if its not well configured.
> But right now, you need a human being to jump onto the user's computer and fire up an assortment of random tools in mystical combinations and use human intuition to divine what's going on, like a god damn Networking Gandalf
not really, assuming you have the right fabric, its nowhere near as hard as that. Plus you seem to be forgetting that there is more to the network than TCP. There is a whole physical layer that has lots of semantics that greatly affect how easy it is to debug higher levels.
The paper argues the in '3.1 Stream orientation' section, that stream orientation is a problem for TCP, and says that most apps send messages instead, and the better protocol should handle messages, natively, etc. Which is a good point I think.
But back to TCP. What do you do, if you need to send Messages between applications in TCP? Preferably those Messages would be encrypted also.
You could make up your own protocol, but you probably would rather not! So you use something that is readily available, and does messages, encryption, etc. Would be nice if there were also a ready to use load balancers, caches, tools to debug it, etc
Now, what would be such a protocol.
Why HTTPS, of course.
So I kind of think that the lack of a low level Message Protocol has lead us, as an industry, to coalesce these features bit-by-bit on top of HTTP. It's not perfect by any means, but it does the job.
Nobody forces them though. It would be much easier to publish a standard port number mapping than to develop a (or multiple) new protocols. Now you just need to motivate people to use it.
I would affirm that. It's imho true for almost everything in IT tech.
How computers "work" today is just pure madness when looked anyhow closer.
Everything's a result of some "historic accidents" back in the days, and from that the usual race to the bottom caused by market powers.
Nobody is willing to touch any of the lower layers no matter how crazy they are form today's viewpoint. We just shovel new layers on top to paper over the mistakes of the past. Nothing gets repaired, or actually what would be more more important, rethought form the ground up in light of new technological possibilities and changed requirements.
I understand from the economic standpoint how this comes. But I'm also quite sure we didn't make any fundamental improvements in the last 50 years of computing.
That's a very bad sign when everything in a field that's not even really 100 years old is frozen in time since 50 years because everything's so fragile and complex that fundamental changes aren't possible. This looks like a text book example of a house of cards…
Given how vital IT tech is to modern life I fear that this will crash at some point in the worst way possible.
And even if it won't crash, which is really strongly hope, we will never have nice things again as nothing of the old rotten things can be reasonably changed.
What's kind of hilarious about this paper is, these are just the network-layer problems! It completely ignores that the "port number" abstraction for service identification has completely failed due to the industry glomming onto HTTP as some sort of universal tunnel encapsulation for all application-layer protocols, rather than a hypertext locator. And then there's all the non-backend problems!
And that's just TCP. We still lack any way to communicate up and down the stack of an entire transaction, for example for debugging purposes. We should have a way to forward every single layer of the stack across each hop, and return back each layer of the stack, so that we can programmatically determine the exact causes of network issues, automatically diagnose them, and inform the user how to solve them. But right now, you need a human being to jump onto the user's computer and fire up an assortment of random tools in mystical combinations and use human intuition to divine what's going on, like a god damn Networking Gandalf. And we've been doing it this way for 40+ years.