News for hackers. Stuff that matters.
Re:UDP vs TCP by mjevans on Wednesday December 28, 2016 @07:34PM (#13274885)
The main difference between TCP and UDP, as this programmer discovered, relates to quality of realtime service.

Times to use UDP over TCP

  * When you need the lowest latency
  * When LATE data is worse than GAPS (loss of) in data.
  * When you want to implement your own form of error correction to handle late/missing/mangled data.
TCP is best when

  * You need all of the data to arrive, period.
  * You want to automatically make a rough best estimate use of the available connection for /rate/ of transfer.
For a videogame, having a general control channel to the co-ordination server in TCP is fine. Having interactive asset downloads (level setup) over TCP is fine. Interactive player movements /probably/ should be UDP. Very likely with a mix of forward error correction and major snapshot syncs for critical data (moving entity absolute location, etc).
Re:UDP vs TCP by Negitivefrags on Thursday December 29, 2016 @03:16AM (#13276256)
People talk like most games use UDP, but they don't.

Pretty much the only genre that does is First Person Shooters.

MMOs (like world or warcraft) MOBAs (like Dota) RTSs (like Starcraft) Action RPGs (like Diablo)

All of these are action games and use TCP.

In most genres people would rather have the simulation pause when there is packet loss, and then run fast to catch up rather than have unreliable data about player locations.

Due to the large number of geographical locations you can have your servers in these days, it's common for most of your players to have pings <30ms. With that kind of latency and a few tweaks, it's possible for a single packet lost to be recovered very quickly with TCP.

Re:UDP vs TCP by the_mitsuhiko on Thursday December 29, 2016 @06:04AM (#13276708)
> MMOs (like world or warcraft) MOBAs (like Dota) RTSs (like Starcraft) Action RPGs (like Diablo)

Dota 2 definitely does not use TCP. The Starcraft games I assume can also work with TCP but I doubt that they use it as a primary choice of communication. I would love a source for this.

Re:UDP vs TCP by luuse on Thursday December 29, 2016 @06:46AM (#13276820)
Starcraft uses a mix of TCP and UDP[1]. I can't find the link now but I remember something about a late switch from TCP to UDP in the beta of the first StarCraft 2 game and some pretty unstable network gameplay while they were tweaking the error handling.

1. https://us.battle.net/support/en/article/300479

Re:UDP vs TCP by theandrewbailey on Thursday December 29, 2016 @08:26AM (#13277116)
Most sites that I come across say that Brood War primarily uses UDP, along with most other similar vintage Blizzard games.

http://wiki.teamliquid.net/starcraft/Port_Forwarding

Re:UDP vs TCP by fosco on Thursday December 29, 2016 @11:34AM (#13278225)
starcraft II uses TCP and UDP. I have verified this by reviewing my firewall, I have not gone as far to take pcaps and see what packets are doing what but I thought this was relevant enough to say.

as the parent mentioned -- UDP where time is more important than reliability. I disagree with his last comment on building your own reliability -- use TCP for that.

tcp/1119, udp/dynamic (listed app definitions for app-based firewall)

some additiona blizzard info --> https://us.battle.net/support/en/article/configuring-router-...

Re:UDP vs TCP by drawkbox on Thursday December 29, 2016 @07:56AM (#13277002)
True but you can create a reliable UDP implementation and get the same TCP features you need (resending/ordering/ACK). Many games use UDP and then some form of reliable UDP where important messages are ACK'd (game start, game end, global events etc).[1] Almost every multiplayer game I have worked on has a flavor of this, where there is a bit or flag on important messages that need reliable tracking over UDP. Everything else is just broadcast like player positions, local action and more where packets can be missed and extrapolated/interpolated, predictive methods and lag compensation.

There can also be some packet loss increase when using TCP mixed with UDP[2]. Most games built recently probably use more RUDP than TCP. But when you need to use any UDP at all you may as well just go reliable UDP. TCP definitely works for non-realtime games and turn based especially but if you ever want some non essential systems updating where packet loss is ok (maybe world physics, or events that are just decorative/fun) then reliable UDP is best in my opinion.

SCTP was supposed to fix much of this but really reliable UDP does almost everything you need for realtime and not, so if investing in a network lib/tech that is the best choice.

enet[3] is a popular reliable UDP framework that was the base or basis for many others. RakNet[4] also has some nice implementations of it. There are many others now but some larger common systems based their systems on these, Unity is one.

[1] https://en.wikipedia.org/wiki/Reliable_User_Datagram_Protoco...

Characteristics of UDP Packet Loss: Effect of TCP Traffic

[2] http://www.isoc.org/INET97/proceedings/F3/F3_1.HTM

[3] http://enet.bespin.org/

[4] https://github.com/OculusVR/RakNet

Re:UDP vs TCP by jeremiep on Thursday December 29, 2016 @11:14AM (#13278092)
This is terrible advice. These games use TCP because they can get away with the latency and often have game designs where latency is a non-issue (300ms in WoW won't ruin your raids or quests). Latency spikes are also very noticeable in these games. Starcraft adds latency locally to your inputs so they're played on all computers simultaneously.

I'd argue that none of these games require time-critical data and often have latencies much higher than they would with UDP. None of these games have networked physics either, TCP breaks down badly in that case. I'd also argue that great network developers come in very short supply and therefore most companies are left to their own misconceptions. (Then again, most game developers grossly overestimate their own skills, nothing new here :D)

I don't know where you get that "people would rather have the simulation pause when there is packet loss, and then run fast to catch up" idea but my experience is the exact opposite.

"Unreliable data about player locations" is a myth. Its just as "unreliable" in TCP with the difference being that in TPC the protocol will arrange for the packet be resent; at which point its no longer relevant to the current frame - you've wasted CPU and bandwidth, crippling your game designers in the process, often without realizing so. You rarely ever want a reliable world state, just reliable actions.

With UDP you actually rely on unreliability to only ever use the latest game state; actions you want reliable or ordered can be implemented very easily on top of UDP or left in TCP. Client-side prediction takes care of dropped frames such that missing a datagram (and therefore that frame's player positions) isn't noticed by the player. A delayed TCP packet will do the exact same, but waste CPU/bandwidth in the process.

I've seen my share of "senior" network programmers who couldn't make a networked engine that is actually stable. I can understand why some teams look at TCP and naively claim "problem solved!" But it doesn't make TCP the best choice because you can appeal to authority with big titles using it :)

Re:UDP vs TCP by cma on Thursday December 29, 2016 @03:47AM (#13276340)
Not doubting, I see some other sources say the same thing, but why do you need to open UDP ports for WOW?

https://us.battle.net/support/en/article/firewall-proxy-rout...

Is it something that changed over time and that page is out of date? Is it just doing TCP over UDP for some esoteric cases?

Re:UDP vs TCP by Negitivefrags on Thursday December 29, 2016 @04:00AM (#13276383)
I don't know why it lists all three ports under both UDP and TCP for WoW. WoW uses 3724 over UDP for it's voice chat, but the other ports are only used with TCP.
Re:UDP vs TCP by chaotic-good on Thursday December 29, 2016 @03:23AM (#13276272)
>> In most genres people would rather have the simulation pause when there is packet loss, and then run fast to catch up rather than have unreliable data about player locations.

I thought that modern games use interpolation to compensate for this, because network lags are always present in WAN.

Re:UDP vs TCP by Negitivefrags on Thursday December 29, 2016 @03:26AM (#13276280)
You can't use interpolation to compensate for late packets. Interpolation is generating data between two data points that you have.

You are talking about extrapolation which is the prediction of future data from previous data points.

Once again, First Person Shooter games will tend to do extrapolation, but few games of other genres do that kind of thing.

Re:UDP vs TCP by jwatte on Thursday December 29, 2016 @08:46AM (#13277198)
It's easier and more robust to do NAT traversal with UDP than with TCP. The Xbox game network services implement TCP within the Microsoft domain on top of UDP!
Re:UDP vs TCP by dxhdr on Thursday December 29, 2016 @12:56PM (#13278780)
Dota 2 and Starcraft 2 both use UDP.
Re:UDP vs TCP by h1d on Saturday December 31, 2016 @02:00PM (#13292525)
Only if all players are from close regions. Will still take 100-200ms for cross continent.
Re:UDP vs TCP by lobster_johnson on Wednesday December 28, 2016 @08:43PM (#13275138)
I always wondered why nobody used UDP for bulk data transfer: Ship all the chunks over with a sequential number identifying each, then when you've reached the end, have the client request any lost packets, then repeat the cycle until all are transferred. After all, the client doesn't need everything in sequential order if it knows the size, and it can deal with holes as long as they're eventually repaired.

This gives you (in theory, I've never tried it) better throughout, and you get the equivalent of TCP's retransmission without the window semantics, since you move all the retransmissions to the end. On the other hand, it's probably bad for everyone else, since there's no congestion control.

Re:UDP vs TCP by kevhito on Wednesday December 28, 2016 @09:23PM (#13275286)
> ship all the chunks over with a sequential number

Except that if you ship them too fast, they will at best simply build up in a buffer somewhere (e.g. near a bottleneck link), or more likely they will overflow causing many of your packets to be lost. So not only did the recipient not get them (and you have to figure this out and resend them later), but you also wasted a bunch of resources sending packets only to have them dropped.

And if you send them too slow, you are leaving some potential capacity on the table. So you need to send them fast, but not too fast.

So you add some additional logic to implement flow control and congestion control. And that's TCP, pretty much.

Re:UDP vs TCP by ajross on Wednesday December 28, 2016 @10:18PM (#13275429)
Exactly. The core feature provided by TCP is not reliability, that is trivially easy. What TCP gives us is robust flow control that can detect and adapt to current network capacity.
Re:UDP vs TCP by snupples on Thursday December 29, 2016 @01:47AM (#13276046)
CNN has a video transfer tool for b2b subscribers that uses multiple UDP streams to increase transfer speeds. Instead it saturated and brought down our company firewalls, which were older and our main bottleneck. All for just a potentially few more megs of speed on TB sized videos. Gotta be careful with UDP.
Re:UDP vs TCP by Dylan16807 on Thursday December 29, 2016 @02:25AM (#13276127)
That just sounds like your firewalls are made of pentium pros and duct tape. TCP can oversaturate a connection too, and a firewall should be able to handle large packets at line rate.
Re:UDP vs TCP by snupples on Friday December 30, 2016 @04:36PM (#13287670)
Yeah the firewalls were ancient and already scheduled for an upgrade. This helped accelerate the schedule for sure. The company had about 800 employees doing all kinds of things, watching Netflix, watching videos on YouTube and Facebook, downloading whole ISO's, not to mention many business processes transferring files around, but the second one person started using this video downloader tool it brought everything to a crawl.
Re:UDP vs TCP by refset on Thursday December 29, 2016 @11:31AM (#13278205)
> a video transfer tool for b2b subscribers that uses multiple UDP streams

That sounds like FASP (http://asperasoft.com/technology/transport/fasp/)

Re:UDP vs TCP by snupples on Friday December 30, 2016 @04:31PM (#13287643)
I believe something like that was the underlying tech. I researched it at the time but I just can't remember 100% now, although I think that was it. It actually was implemented via a Java applet in the browser, so there was no way we could block or restrict a specific exe with group policy. The firewalls were already scheduled for replacement/upgrade, so for a few weeks we ended up just watching traffic and slapping wrists if more than one person ever tried to use the tool at the same time.
Re:UDP vs TCP by Dylan16807 on Thursday December 29, 2016 @01:17AM (#13275947)
A robust flow control that's tuned for bulk downloads, and badly adapted for small real-time streams.
Re:UDP vs TCP by ajross on Thursday December 29, 2016 @09:20AM (#13277374)
Sure. And not even the best known flow control mechanism. But nonetheless that is the important innovation of TCP, not reliability. The post by lobster_johnson upthread implied that you could use UDP for "bulk data transfer", which basically doesn't work without reinventing something akin to TCP.
Re:UDP vs TCP by ajosh on Wednesday December 28, 2016 @11:44PM (#13275672)
There is actually such a protocol - UFTP. It was designed for high-latency connections such as satellite links. It can use multicast to allow multiple recipients over such a link. If memory serves, it was designed by Gannet to help transfer news content nationally for USA Today.

I've tested it on "long fat pipes" when evaluating it for a project a number of years ago. The numbers don't sound that impressive now. I was able to saturate the 100Mbps NIC of the slower server in the transfer. The transfer was from Northern VA to LA over the Internet. This was from data center to data center and the network capacity was such that I knew that the transfer rate would not cause any serious packet loss.

Using simple TCP over the same link, same computers was limited to around 40Mbps. More modern TCP implementations have a larger window and are able to better saturate high-throughput, high-latency links.

Re:UDP vs TCP by acomjean on Wednesday December 28, 2016 @08:51PM (#13275168)
UDP messages are limited to 64K. It can be a pain to reassemble. lots of ugly corner cases.

We had need to send one type of message split due to size with udp. We were on the same subnet sending and receiving so it worked almost always. We were also using multicast.

Its very very hard to figure out whats going on when things don't work ( You don't know if the control messages are getting through, when you ask again or if the message just isn't being sent). At some point you are reinventing tcp/ip.

I always liked the way udp messages are received though, all or nothing. Not the byte stream that is tcp which needs to be broken into messages manually.

Re:UDP vs TCP by lobster_johnson on Wednesday December 28, 2016 @09:06PM (#13275226)
Picking a smallish, fixed chunk size should work, and it could even be made to self-regulate. Reassembly is trivial -- allocate the file first, write chunks to their right location. And for the control stuff (i.e. requesting chunks) you just use a TCP connection. UDP for the data only.
Re:UDP vs TCP by jerf on Wednesday December 28, 2016 @09:14PM (#13275252)
If you do the obvious work to convert UDP to a reliable streaming mechanism, you will generally more-or-less reinvent TCP, only probably more poorly, and would probably be better off just using TCP. UDP is a good choice when you have options or requirements that TCP can't accommodate. For instance, the classic case of streaming audio has both; it both requires timely delivery and has some very sophisticated techniques for dealing with missing packets with a lot of local intelligence about human perception and such built into the codecs. If you can drop packets without issue, if future packets completely supercede past packets (FPS network gaming for instance), there's a lot of possibilities, but if you're just going to recreate reliable streaming you probably be net losing on account of the effort spent doing that instead of something useful, and bugs in the implementation.
Re:UDP vs TCP by theWatcher37 on Thursday December 29, 2016 @01:42AM (#13276033)
Why are so many comments here ignoring the purpose of the article?

The purpose of this is for games, not web traffic or torrent2.0.

In real time games where you're syncing physics states or player locations data >200ms isn't just useless it's actually negative.

This article is 2+ years old, and is the foundation I suspect for libyojimbo which hybridizes UDP low latency communication for large player real time games and reliable messaging (basically custom TCP?) for less time critical things.

Re:UDP vs TCP by jerf on Sunday January 01, 2017 @07:42PM (#13298305)
I don't know, but that doesn't have anything to do with my comment, which is entirely technical. Perhaps you meant to put it somewhere else?
Re:UDP vs TCP by vvanders on Wednesday December 28, 2016 @08:45PM (#13275146)
You've kinda answered your own question there, congestion control becomes a big issue(also MTU discovery to a lesser degree).

Games sit in this happy space where the data rate is ~10-20kb at most so they don't need to worry about congestion control as much.

Re:UDP vs TCP by user5994461 on Thursday December 29, 2016 @01:50AM (#13276054)
Additional information: The source engine (valve) has been limited to 30 kb/s for more than a decade.
Re:UDP vs TCP by ak217 on Wednesday December 28, 2016 @09:59PM (#13275386)
There are many protocols that use this approach, Aspera FASP is possibly the most advanced one. It uses a TCP control channel to manage the rate on the UDP data channel.
Re:UDP vs TCP by chaosphere2112 on Wednesday December 28, 2016 @10:13PM (#13275417)
On LAN, this is basically how Symantec Ghost works; it multicasts the image over UDP, and each client figures out what they missed. Those chunks get retransmitted at the end.
Re:UDP vs TCP by justin66 on Wednesday December 28, 2016 @09:14PM (#13275254)
> I always wondered why nobody used UDP for bulk data transfer

That would be odd. It's not true. There's TFTP all the way up to UDT.

Re:UDP vs TCP by gregschlom on Wednesday December 28, 2016 @09:25PM (#13275296)
Although to be fair, given that with TFTP each packet must be acknowledged before the next one is sent, you're actually achieving a way worse throughput than TCP. Not a very good example on how to use UDP efficiently IMO.
Re:UDP vs TCP by justin66 on Wednesday December 28, 2016 @10:03PM (#13275396)
UDT is a better example.

http://udt.sourceforge.net/

Re:UDP vs TCP by blackflame7000 on Wednesday December 28, 2016 @09:00PM (#13275206)
What it really comes down to is file block size granularity. TCP is simply using a much finer block size (~1500 bytes) which means a higher likelihood of arriving than for a large UDP packet. Furthermore, you'd still have to implement your own checksumming algorithm to ensure data integrity as UDP doesn't require a packet checksum for IPv4. I suppose you could offload file-block integrity validation to another thread so you can receive and verify in parallel, but at some point you have to checksum every block anyways which means the full file must be processed to figure out where corruption might be before sending retry requests for the missing packets.

I see what you're getting at by trying to avoid processing the packets in order if it wastes time, but ultimately I think its a micro-op that introduces other bottlenecks. That's why bit-torrent still typically uses TCP even though they could potentially receive file parts in any order.

Re:UDP vs TCP by SomeStupidPoint on Wednesday December 28, 2016 @10:03PM (#13275394)
But bittorrent breaks the files in to chunks it can receive out of order and sends those over TCP, so TCP only preserves ordering within a chunk. It might even get chunks from several different people.

The real answer is that chunking over TCP is efficient enough that little is gained by chunking over UDP.

Re:UDP vs TCP by blackflame7000 on Thursday December 29, 2016 @09:14AM (#13277349)
Yes what I'm saying is that bit torrent uses TCP to segment fileblocks instead of using UDP to send the file block as one large datagram
Re:UDP vs TCP by lend000 on Thursday December 29, 2016 @12:04AM (#13275726)
That sounds like a TCP congestion control mechanism, except worse! TCP leaves much to be desired in terms of performance, largely because their congestion control fails miserably on lossy links (TCP RENO). There are better backoff mechanisms that can be added on many distros, like Westwood. And then there are altogether more efficient protocols like QUIC. But designing one for general purposes from UDP is easier said than done.
Re:UDP vs TCP by scurvy on Wednesday December 28, 2016 @09:09PM (#13275235)
NFS on Linux defaulted to UDP for what seems like a really long time. There were perf issues with TCP (can't remember what they were).
Re:UDP vs TCP by mullen on Wednesday December 28, 2016 @09:24PM (#13275289)
NFS used to default to UDP for the same reasons that games still would use UDP, it just needed the data as quickly as possible since it was blocking IO. Now a days, we don't need UDP for NFS because networks are a lot faster and the advantages of TCP start to over rule UDP.
Re:UDP vs TCP by fulafel on Wednesday December 28, 2016 @11:56PM (#13275706)
The original reason was that using UDP was much faster CPU-wise back in the 80's. There was a requirement to go fast because it was used for transferring data in bulk, not events like games.

When Linux came around it just followed suit. TCP support came to NFS only later with other bells and whistles in later NFS versions, and Linux stuck with basic NFSv2 for a good while.

NFS was designed for local-area networks, so it didn't need to worry about working over the Internet.

In addition to the network being much faster relative to CPUs back then, the TCP stacks hadn't seen the tuning they have now. Making TCP fast came later.

Re:UDP vs TCP by tedunangst on Wednesday December 28, 2016 @09:32PM (#13275314)
NFS (was) stateless and idempotent. Lose a packet? Send it again! Which is mostly what you want, since there can be dozens of processes performing IO and they're all independent. Last thing you want is head of line blocking.
Re:UDP vs TCP by user5994461 on Thursday December 29, 2016 @01:48AM (#13276050)
> I always wondered why nobody used UDP for bulk data transfer

1) Bulk data transfer doesn't work over UDP.

There is no flow control whatsoever with UDP and no guarantee, it sends stuff without distinction between a 56k uplink and a 10Gigabit LAN. The packet loss and lack of reliability are atrocious when you send non negligible amount of data.

2) You'd need to create a protocol on top of UDP to handle the basics (controlflow/error/retransmission), that's equivalent to re-inventing TCP. Don't re-invent your own TCP, just use TCP ;)

Re:UDP vs TCP by cthalupa on Thursday December 29, 2016 @03:34AM (#13276309)
>1) Bulk data transfer doesn't work over UDP.

Tell that to all of the people that use TsunamiUDP to do bulk data transfer. It uses TCP as a control protocol, but the data transfer is all UDP.

Re:UDP vs TCP by paulddraper on Friday December 30, 2016 @11:42AM (#13285265)
It sounds like they would agree.

Bulk data transfer doesn't work over UDP, but it can work over UDP and TCP.

Re:UDP vs TCP by cthalupa on Saturday December 31, 2016 @02:44PM (#13292767)
The data transfer itself is happening over UDP.
Re:UDP vs TCP by astrobe_ on Thursday December 29, 2016 @08:07AM (#13277047)
UDP is usable for this when used to transport a file transfer protocol. I've seen for instance Kermit over UDP. Basically, if you transport data using a protocol that already takes care of integrity, using TCP is more or less redundant and just adds annoyances (connection management).
Re:UDP vs TCP by orasis on Thursday December 29, 2016 @12:06AM (#13275729)
Back in the day of hard drives, the random access I/O would be nasty and you'd have to be careful about creating sparse files, so you'd end up double writing the missing packets. you'd also still need congestion control so might as well just use TCP to retransmit.
Re:UDP vs TCP by ra1n85 on Wednesday December 28, 2016 @09:01PM (#13275207)
Bad for everyone else, and it's up to the implementation to handle flow-control. Congestion-control handles network related loss/latency/load. Flow-control helps with receiver related loss/latency/load.
Re:UDP vs TCP by Franciscouzo on Thursday December 29, 2016 @10:48AM (#13277884)
A lot of the internet's bulk data transfer uses UDP, torrents.
Re:UDP vs TCP by lobster_johnson on Thursday December 29, 2016 @02:03PM (#13279228)
BitTorrent is mostly TCP these days, though, as pointed out elsewhere in this thread.
Re:UDP vs TCP by morgante on Wednesday December 28, 2016 @10:13PM (#13275416)
In practice, I think you'd end up having to reimplement most of TCP on top of UDP anyways.
Re:UDP vs TCP by nrjdhsbsid on Wednesday December 28, 2016 @11:37PM (#13275648)
This isn't really true, or at least not the main reasons to choose UDP or TCP .

In modern days UDP is really only used in situations where you can afford packet loss and CPU is expensive, or when you need to bypass the built-in behavior of TCP on your OS. It's not as much about latency or data loss, usually UDP is used in places where processing power is expensive or load is too extreme to justify TCP.

Sometimes, like with DNS, it's better to just resend to request than burden a massively parallel server with maintaining TCP state and checksums.

The main issue I have with the above comment is that UDP absolutely does not guarantee better latency or have higher priority than TCP packets. The packets usually queue in the NIC buffers and downstream identically regardless of which protocol you use. The main draw of TCP besides data and ordering guarantee (which are CPU costly) is that (depending on your operating system) it follows some kind of automatic rate limiting algorithm be it TCP Vegas(Linux, delay based) or Reno (windows, loss based). These algorithms attempt to limit your connection "stream" to the line rate automatically and play somewhat nicely with each other on large scale.

In contrast, the main problem with using UDP is figuring out a smart way to rate the limit to avoid excessive packet loss. In some protocols like DNS anycast type things where only raw throughout matters, who cares if responses drop. Other times, like with LEDBAT or Microsoft's BITS, your main reason for using UDP is to roll your own rate limiting protocol, since otherwise you're stuck with what the OS gives you on TCP.

Re:UDP vs TCP by banachtarski on Thursday December 29, 2016 @12:03AM (#13275722)
I'm not sure I follow your argument. Say I want to communicate from player A the position of player A to player B. The position is really only relevant for a 200 ms time frame or so (the game might support teleports, jumping, dashing, etc). In a TCP setting, even if a packet is old (>200ms), I have no choice but to send it regardless, and all new packets must queue behind it. The whole point behind using UDP for games is that I can control what the resend policy is, which determines what ends up in the buffers you are referring to in your post. TCP gives me not only little to no choice in the matter, but often does the worst thing in a critical situation, especially in a game where input latency is 6 ms and the tolerance for network latency is <50 ms.
Re:UDP vs TCP by nrjdhsbsid on Thursday December 29, 2016 @12:26AM (#13275791)
You're right in that UDP will let you send the most updated data without requiring everything to be sent. My main issue was that UDP does not give you lower latency. TCP algorithms are extremely aggressive to the point I doubt you could get more recent packets on the wire with UDP.

The problem is, the OS throws everything into the same buffer, and unless you're clairvoyant you won't be able to skip sending old packets into the buffer until it's too late. Basically by the time you know the data is outdated it's already on the way to the NIC buffer and you can't stop it, making TCP and UDP equal in this regard

Re:UDP vs TCP by Dylan16807 on Thursday December 29, 2016 @12:34AM (#13275817)
You're only considering the case when no packets are lost.

Once TCP drops a packet, data sits in the receiving buffer getting older and older instead of being delivered to the program.

Re:UDP vs TCP by nrjdhsbsid on Thursday December 29, 2016 @05:20AM (#13276598)
It's way more rediculous than you think. OS network stacks try to do the reasonable thing in most cases if possible. In the case of Linux TCP Vegas is very unlikely to drop any packet since the main algorithm is RTT based not drop.

Red and hopefully codel can combat buffer bloat to some point but in the end all packets go to the same place

Re:UDP vs TCP by barrkel on Thursday December 29, 2016 @07:20AM (#13276906)
I see packet loss and spurious resends (lost acks) all the time on WiFi in certain locales. You are massively underplaying packet loss imo.
Re:UDP vs TCP by zurn on Thursday December 29, 2016 @06:23AM (#13276759)
I think he meant the situation where the network drops a TCP segment, leading to head-of-line blocking.
Re:UDP vs TCP by hueving on Thursday December 29, 2016 @03:12AM (#13276248)
Waiting for retransmission of a lost segment in front of you in the queue is massive latency.
Re:UDP vs TCP by the_mitsuhiko on Thursday December 29, 2016 @06:05AM (#13276711)
TCP does head of line blocking so your entire argument is just wrong. TCP by design cannot give you the latency guarantees that UDP can.
Re:UDP vs TCP by AceJohnny2 on Thursday December 29, 2016 @12:56AM (#13275883)
I'm getting the impression that you're mostly only taking the endpoints into account. There's tons of stuff happening in between the peers: having worked on specialty routing equipment, I can add that the prevailing philosophy was that generic UDP packets were lower quality-of-service than TCP. That is, if the router was under high load, UDP would be dropped before TCP.
Re:UDP vs TCP by TwoBit on Thursday December 29, 2016 @01:20AM (#13275962)
Windows doesn't use Reno. It uses Compound TCP, which is somewhat like Vegas.

https://en.m.wikipedia.org/wiki/Compound_TCP

Re:UDP vs TCP by exDM69 on Wednesday December 28, 2016 @07:42PM (#13274915)
Another article on this site addresses TCP plus UDP mix. The tl;dr is that they should not be used at the same time because TCP retransmission will interfere with UDP (and routers give TCP priority).

As long as TCP happens outside of the "hot" loop, it's fine.

Re:UDP vs TCP by wtallis on Wednesday December 28, 2016 @08:03PM (#13274993)
> (and routers give TCP priority).

What routers? Is this something that stupid routers did in the '90s and people are still worried about, or are modern routers actually behaving this way? (And if so, why?)

TCP retransmission interfering with UDP sounds like something that shouldn't be a problem on any sane network, especially if the application authors are smart and use something like LEDBAT for their TCP traffic when they're also doing latency-sensitive UDP communication.

Re:UDP vs TCP by fulafel on Thursday December 29, 2016 @12:32AM (#13275810)
The internet is generally built on the implicit contract of "TCP-friendly flow control". If your homegrown protocol backs off slower than TCP, some router configurations will drop your packets first.

There is no general policy against non-TCP packets of course, as lots of important internet services are non-TCP. Just flows that don't respond to congestion.

It's not a vendor specific thing, all the major router vendors provide ways of doing this.

(Note: router != your home NAT box)

Re:UDP vs TCP by wtallis on Thursday December 29, 2016 @12:55AM (#13275879)
> If your homegrown protocol backs off slower than TCP, some router configurations will drop your packets first.

I think that's only if you're being less responsive to congestion signals (drops, ECN marks), and you're using more than a fair share of bandwidth or trying to use any available bandwidth. Fixed but low rate flows (eg. VoIP) shouldn't be penalized until the link is congested with so many flows that the VoIP is trying to use more than 1/N of the bandwidth.

Re:UDP vs TCP by vvanders on Wednesday December 28, 2016 @08:40PM (#13275125)
You'd be shocked at the things home routers do. There's a reason most people don't try to implement NAT traversal and use common middleware.

Since UDP isn't as common of a protocol I can certainly see some routers treating it a second class traffic when it comes to packet shaping.

Re:UDP vs TCP by wtallis on Wednesday December 28, 2016 @08:47PM (#13275155)
I'm well aware of how dumb home routers can be, but a general de-prioritizing of the transport protocol that DNS uses is both really dumb and pretty easy to detect. Are you aware of any router vendors that have actually shipped such a configuration? (Or for that matter, any consumer router that has shipped with any prioritization rules enabled out of the box?)
Re:UDP vs TCP by kalleboo on Thursday December 29, 2016 @12:52AM (#13275865)
Every home network I've seen ends up using the home router itself as a forwarding DNS server as configured through DHCP.

That said, I doubt home routers are de-pritoritising UDP to any extent, or it would be a big topic of discussion among gamers.

Re:UDP vs TCP by macintux on Wednesday December 28, 2016 @09:17PM (#13275265)
DNS strikes me as something relatively painless to deprioritize. Small packets, not all that latency-sensitive for most use cases.
Re:UDP vs TCP by wtallis on Wednesday December 28, 2016 @09:32PM (#13275312)
DNS isn't VoIP, but most DNS traffic is very much latency sensitive. When you click a link to a typical modern web page, you trigger multiple HTTP connections to load the page and its many resources hosted on dozens of domains. The DNS lookups are on the critical path for all of those requests.
Re:UDP vs TCP by macintux on Wednesday December 28, 2016 @09:37PM (#13275327)
Yes, but the HTTP requests themselves dominate the overall load time, even if DNS takes a bit longer. Plus there's caching to reduce somewhat the need for the DNS requests.

(Of course, I'm pulling all of this out of my nether regions without a lot of thought, and you may well be right, the sheer volume of such requests might lead to problems.)

Re:UDP vs TCP by wtallis on Wednesday December 28, 2016 @09:51PM (#13275369)
It looks like on Windows a typical DNS retry timout is 1 second, with backoff if multiple retries are needed. If a DNS packet needed for loading a web page gets dropped, it's very likely to increase the overall page loading time. Re-ordering DNS to be delivered only when there's a lull in the large TCP packets would almost serialize the loading of resources from different domains.

DNS caching is a good thing, but it certainly doesn't eliminate this problem. Web browsing still produces a lot of DNS requests, and any cache upstream of the bottleneck (ie. any cache operated by your ISP rather than in your own router) doesn't help against loss during congestion.

Re:UDP vs TCP by kijiki on Wednesday December 28, 2016 @10:47PM (#13275507)
That would be a disaster.

Without active queue management, a single bulk upload can break DNS. When the bottleneck link (usually your cable/DSL modem) is saturated by the bulk upload, it will start dropping new packets that come in while the queue is full.

When a single packet worth of space frees up, the chances are that the bulk flow will instantly take it, such that nearly all outbound DNS packets get dropped, and DNS queries time out.

The solution is active queue management (CODEL is a good choice) at every bottleneck.

Re:UDP vs TCP by jlgaddis on Wednesday December 28, 2016 @10:22PM (#13275444)
I'm a network engineer and I think this is the first time I've ever heard the claim that "routers give TCP priority" (ICMP is often de-prioritized but that's on the control plane).

Exactly which routers are alleged to be doing this?

Re:UDP vs TCP by placebo on Wednesday December 28, 2016 @09:05PM (#13275222)
There are quite a few "reliable UDP" implementations out there - I wonder if the same considerations would hold. Guess it might depend on the implementation of reliability using UDP.
Re:UDP vs TCP by user5994461 on Thursday December 29, 2016 @12:43AM (#13275841)
Damn. I disagree with all of your points.

Use UDP only if you satisfy all the following conditions:

  * Don't care about loosing data
  * Don't care about receiving data out of order
  * Not sending stream of data but individual messages ( < 580 bytes)
  * Don't care about the other side of the connection being dead or terminated, and not knowing about it
Re:UDP vs TCP by Dylan16807 on Thursday December 29, 2016 @12:55AM (#13275878)
UDP doesn't cause things like not knowing if the other end is gone. You're being kind of ridiculous.
Re:UDP vs TCP by user5994461 on Thursday December 29, 2016 @01:00AM (#13275899)
UDP is connectionless, it just sends data to <somewhere>. Doesn't care if that somewhere is online, or available, or exists at all.
Re:UDP vs TCP by Dylan16807 on Thursday December 29, 2016 @01:14AM (#13275943)
If you don't check for returns, TCP isn't going to help either.

The requirements you listed don't make sense as written. Nobody would ever use UDP.

Re:UDP vs TCP by user5994461 on Thursday December 29, 2016 @01:58AM (#13276072)
TCP establishes a connection when it starts and it has build-in timeouts (and various mechanisms) to detect dead connections. TCP will signal to the application if the destination is unavailable or lost.

The requirements make sense and the common usages for UDP fit these requirements.

It's correct that [almost] nobody ever uses UDP. It's a niche protocol for a very few specific use cases, one of which is real-time FPS/RTS/MOBA games (which fits all the criteria I listed).

Re:UDP vs TCP by raarts on Friday December 30, 2016 @01:52AM (#13282988)
> it's correct that [almost] nobody ever uses UDP

Skype, Facetime, WhatsApp, Hangouts, Telco VoIP, Facebook Live, YouTube Live, Surveillance Cams, most VPNs.

Re:UDP vs TCP by the_mitsuhiko on Thursday December 29, 2016 @06:06AM (#13276717)
TCP by itself does not detect dead connections.
Re:UDP vs TCP by y4mi on Thursday December 29, 2016 @10:42AM (#13277847)
while its technically not a timeout, it is stopping the network traffic

the TCP packets require an ACK ever n packets, as defined by the FREQ field. If the ACK isn't received, no further packets are send.

Re:UDP vs TCP by fulafel on Wednesday December 28, 2016 @11:47PM (#13275686)
Comparing TCP and UDP generally is nonsensical.

The sensical comparison would be TCP vs whatever protocol you build or use. It might be your homegrown protocol, but if you have problems with TCP, you should probably look at other ready-made protocols first. For games, there's eg ENet.

(Not to say that TCP is generally out of the question for games, eg. WoW uses it apparently successfully)

Re:UDP vs TCP by skarap on Thursday December 29, 2016 @04:57AM (#13276527)
Nobody ever mentions that UDP has one additional feature that TCP doesn't: preserving of message boundaries.
Re:UDP vs TCP by kmm on Thursday December 29, 2016 @10:01AM (#13277585)
A datagram-like protocol with the reliability of TCP might be nice.

I once implemented a communications protocol that worked with variable-length independent messages over TCP. It felt a little silly pushing distinct datagrams into a stream, that would be chopped up into a packets, sent to another PC, where the OS would then reassemble those packets back into a stream, only to be chopped up again in messages.

Implementing the system for chopping up the stream really felt like it should have been done by a lower level as well, not in an application

Re:UDP vs TCP by moosingin3space on Saturday December 31, 2016 @03:15PM (#13292918)
Seems to me that ZeroMQ basically implemented this - length-prefixed framing is part of the wire protocol.
Re:UDP vs TCP by fosco on Thursday December 29, 2016 @11:35AM (#13278239)
> * When you want to implement your own form of error correction to handle late/missing/mangled data.

if this is a consideration -- using TCP may be best option.

Re:UDP vs TCP by yaur on Thursday December 29, 2016 @12:17AM (#13275761)
IMO the rule is actually really simple. If the correct thing to do when you lose a packet is to retransmit it use TCP, if it's not use UDP.
Re:UDP vs TCP by 0xcde4c3db on Thursday December 29, 2016 @12:00PM (#13278393)
I think that rule is a little too simple, because it might be correct to retransmit but not to have the retransmission block processing of later packets. I'd say one should probably use TCP when any of the following are true:

1) The correct interpretation of later packets depends on side effects of earlier packets (e.g. commands in an SSH session, OpenGL commands)

2) Raw bytes-per-second throughput is a major consideration (e.g. file transfer, streaming video)

3) The application readily tolerates high latency and jitter (e.g. email, text chat)

Re:UDP vs TCP by problems on Wednesday December 28, 2016 @08:46PM (#13275150)
You can also use something like enet which allows for both reliable and non reliable channels.
Re:UDP vs TCP by disiplus on Thursday December 29, 2016 @03:53AM (#13276362)
also wherever you need real time there is mostly UDP in use, For SIP + RTP, webrtc is a little bit complicated but the real time part is based on UDP. all those have a mechanisms implemented for packet loss and how to handle it.
Re:UDP vs TCP by dabockster on Wednesday December 28, 2016 @08:43PM (#13275141)
Isn't UDP more secure than TCP since it does not implement a sequence number (among other things)?
Re:UDP vs TCP by frutiger on Wednesday December 28, 2016 @08:50PM (#13275163)
Questions like "more secure" and "less secure" only make sense in some specific context with a well-defined threat model: what it is that you want to keep safe from which classes of threats.

For many (if not most) threat models, the lack of a sequence number does not provide adequate security.