I recall my computer networking professor describing this as "Best-effort means never having to say you're sorry". In practice, best-effort does not mean you try your hardest to make sure the message gets from A to B, it means that you made an effort. Router in the path was congested? Link flap leading to blackholing on the order of 50ms before fast reroute kicks in? Oh well, we tried.
Meanwhile, TCP's reliable delivery will retry several times and will present an in-order data stream to the application.
Reliable vs unreliable might be bad terminology, but I don't think best-effort is any better.
My experience with unreliable systems is that they're great something like 95% of the time, and they're great for raw throughput, but there are many cases where that last 5% makes a huge difference.
You can build a 'reliable' protocol on top of UDP, and still not get TCP.
Eg if you want to transfer a large file that you know up front, then TCP's streaming mechanism doesn't make too much sense. You could use something like UDP to send the whole file from A to B in little chunks once, and at the end B can tell A what (numbered) chunks she's missing.
There's no reason to hold off on sending chunk n+1 of the file, just because chunk n hasn't arrived yet.
Congestion control comes to mind -- you don't necessarily know what rate the network supports if you don't have a feedback mechanism to let you know when you're sending too fast. Congestion control is one of those things where sure, you can individually cheat and possibly achieve better performance at everyone else's expense, but if everyone does it, then you'll run into congestive collapse.
> You can build a 'reliable' protocol on top of UDP, and still not get TCP.
I agree -- there are reliable protocols running on top of UDP (e.g. QUIC, SCTP) that do not behave exactly like TCP. You don't need an in-order stream in the described use case of bulk file transfer. You certainly don't need head-of-line blocking.
But there are many details and interactions that you and I wouldn't realize or get right on the first try. I would rather not relearn all of those lessons from the past 50+ years.
Oh, the model I had in mind was not that everyone should write their network code from scratch all the time, but rather that everything that's higher level than datagrams should be handled by unprivileged library code instead of privileged kernel level code.
If speed is an issue, modern Linux can do wonders with eBPF and io_uring, I guess? I'm taking my inspiration from the exokernel folks who believed that abstractions have no place in the operating system kernel.
the unix model is different: it's basically premised on a minicomputer server which would be running a diverse set of independent services where isolation is desired, and where it makes sense for a privileged entity to provide standardized services. services whose API has been both stable and efficient for more than a couple decades.
I think it's kind of like cloud: outsourcing that makes sense at the lower-scale of hosting alternatives. but once you get to a particular scale, you can and should take everything into your own hands, and can expect to obtain some greater efficiency, agility, autonomy.
Exokernels provide isolation. Secure multiplexing is actually the only thing they do.
> and where it makes sense for a privileged entity to provide standardized services. services whose API has been both stable and efficient for more than a couple decades.
Yes, standardisation is great. Libraries can do that standardisation. Why do you need standardisation at the kernel level?
That kind of standardisation is eg what we are doing with libc: memcpy has a stable interface, but how it's implemented depends on the underlying hardware; the kernel does not impose an abstraction.
The closest I can think of is the old FSP protocol, which never really saw wide use. The client would request each individual chunk by offset, and if a chunk got lost, it could re-request it. But that's not quite the same thing.
That said, the typical reason why TCP doesn't send packet N+1 is because its congestion window is full.
There is a related problem known as head-of-line blocking where the application won't receive packet N+1 from the kernel until packet N has been received, as a consequence of TCP delivering that in-order stream of bytes.
Basically, when transmitting a file, in principle you could just keep sending n+k, even if the n-th package hasn't been received or has been dropped. No matter how large k is.
You can take your sweet time fixing the missing packages in the middle, as long as the overall file transfer doesn't get delayed.
"Effort" generally refers to some sort of persistence in the face of difficulty. Dropping a packet upon encountering a resource problem isn't effort, let alone best effort.
The way "best effort" is used in networking is quite at odds with the "best efforts" legal/business term, which denotes something short of a firm commitment, but not outright flaking off.
Separately from the delivery question, the checksums in UDP (and TCP!) also poorly assure integrity when datagrams are delivered. They only somewhat improve on the hardware.
The dropping part isn’t the effort; the forwarding part is.
> the checksums in UDP (and TCP!) also poorly assure integrity when datagrams are delivered.
That’s true, but it’s becoming less of a problem with ubiquitous encryption these days (at least on WAN connections).
Anyway, so the question is, if typical IP forwarding is "best effort" ... what is an example of poor effort, and what exhibits it?
Oversimplified.
Best effort is a dumb term. There’s no effort.
reliable = it either succeeds or you get an error after some time, unreliable = it may or may not succeed
I concur with the people who think "best effort" is not a good term. But perhaps TCP streams are not reliable enough for TCP to be rightly called a reliable stream protocol. As it turned out, it's not really possible to use TCP without a control channel, message chunking, and similar mechanisms for transmitting arbitrary large files. If it really offered reliable streams that would its primary use case.
best effort implies 'no, we're not going to be climb that curve and get try to get to 100% reliability, because that would actually be counterproductive from an engineering perspective, but we're going to go to pretty substantial lengths to deliver your packet'
Reliability is arguably more of a statement about the availability metrics of your underlying network; it doesn’t seem like a great summary for what TCP does. You can’t make an unreliable lower layer reliable with any protocol magic on top; you can just bundle the unreliability differently (e.g. by trading off an unreliability in delivery for an unreliability in timing).
UDP is at-most-once; TCP is exactly-once-in-order.
That's what I thought "unreliable" meant? I can't really tell what misconception you are trying to avoid.
Ultimately, TCP vs UDP per se is rarely the right question to ask. But that is often the only configuration knob available, or at least the only way to get away from TCP-based protocols and their overhead is to switch over to raw UDP as though it were an application protocol unto itself.
Such naive use of UDP contributes to the stigma. If you send a piece of data only once, there's a nontrivial chance it won't get to its destination. If you never do any verification that the other side is available, misconfiguration or infrastructure changes can lead to all your packets going to ground and the sender being completely unaware. I've seen this happen many times and of course the only solution (considered or even available in a pinch) is to ditch UDP and use TCP because at least the latter "works". You can say "well it's UDP, what did you expect?" but unfortunately while that may have been meant to spur some deeper thought, it often just leads to the person who hears it writing off UDP entirely.
Robust protocol design takes time and effort regardless of transport protocol chosen, but a lot of developers give it short shrift. Lacking care or deeper understanding, they blame UDP and eschew its use even when somebody comes along who does know how to use it effectively.
I think the biggest mistake people make with UDP protocols is allowing traffic amplification attacks.
In the end, best-effort is just saying "unreliable" in a fussier way.
>But it does not make UDP inherently unreliable.
Isn't that exactly what it does make it?
If that's not it, then what woud an actual "inherently unreliable" design for such a protocol be? Calling an RNG to randomly decide whether to send the next packet?
> UDP makes its best effort
TCP tries really hard, too, you know.
but, "best-effort" implies that it's doing some effort to ensure delivery, when it's really dropping any packet that looks funny or is unlucky enough to hit a full buffer
i like "lossy", but this is definitely one of the two hard problems
[1] https://en.wikipedia.org/wiki/Best-effort_delivery