News for hackers. Stuff that matters.
Re:Is it just me or is networking really hard? by chetanahuja on Sunday September 13, 2015 @02:53PM (#10212387)
"You can’t even imagine a way that reliability could be implemented on top of UDP that beats TCP? What total bullshit"

This. This "TCP already does it best in every situation" is a common trope usually spouted by people who know nothing about what TCP does and why. The easiest way to get over this misconception is to ask yourself, can you think of a protocol that not only works well over all sorts of networks, all the way from two boxes connected by a 10Gig switch (with 10us ping latency) vs two hosts talking to each other via a satellite link (with perhaps, 2 second ping latency) but is the best option across all those situations?

The truth is that TCP is the common denominator that's available in pretty much every device you would want to use. But in almost every individual use case, you could come up with a far better protocol if you took the time and effort to treat the problem seriously.

Re:Is it just me or is networking really hard? by nubb on Sunday September 13, 2015 @03:02PM (#10212423)
UDP is used when it's designed by network engineers. That's why we have VOIP and video streaming - all are served UDP.

dont get me started on the lack of multicast out there. imagine game servers running multicast. one day.

Re:Is it just me or is networking really hard? by chetanahuja on Sunday September 13, 2015 @03:06PM (#10212442)
Well... those are examples of designing custom protocols on top of UDP. These network engineers, who I assume are specially trained in some secret govt. facility for network commando training, actually have to understand the problem at hand and design something specifically for the problem. Calling these protocols just "UDP" is like saying something like "I use wheels to commute from home to office everyday"
Re:Is it just me or is networking really hard? by efaref on Sunday September 13, 2015 @04:00PM (#10212609)
I would hardly call protocols like [SIP](https://www.ietf.org/rfc/rfc3261.txt) and [RTP](https://www.ietf.org/rfc/rfc3550.txt) "custom". There are many more standard protocols out there than just TCP and UDP. There's even a "better TCP" in the form of [SCTP](https://tools.ietf.org/rfc/rfc4960.txt).

The fact is that TCP was the MVP for reliable streams of data on the 1970s internet, and is woefully inadequate for modern usage, but we're stuck with it (including its oft-disabled parts like Nagle's Algorithm) because it's what's available everywhere.

Game network protocols are pretty easy in comparison with some of the protocols I've encountered in my day job, especially once you've learned the lessons of people like the author of that article.

Re:Is it just me or is networking really hard? by chetanahuja on Sunday September 13, 2015 @06:47PM (#10213043)
'I would hardly call protocols like [SIP]... and [RTP]... "custom".'

The meaning of "custom" in this context was to distinguish it from a full TCP or raw UDP. Not to imply that there's no standard definitions for VOIP protocols. Quite the contrary in fact. The point being made is that UDP is just a substrate to create other protocols that also need to be designed for a purpose. And just calling all these protocols "UDP" is a mistake.

Btw, not to be argue from authority, but to back my words with some context, I'm responsible for at least one such "custom" protocol myself. https://packetzoom.com/blog/

Re:Is it just me or is networking really hard? by jamesblonde on Sunday September 13, 2015 @04:26PM (#10212699)
A lot more traffic than you can imagine comes over UDP. Most Bittorrent traffic (used to be a majority of traffic on the Internet!) comes over LEDBAT, which runs over UDP, https://tools.ietf.org/html/rfc6817. WebRTC by Google is now using UDP for most things, because NAT traversal is not really possible over TCP (due to the 3-way handshake, and the SYN-SYN connection setup not being widely supported by OSes).
Re:Is it just me or is networking really hard? by samplonius on Sunday September 13, 2015 @03:57PM (#10212599)
Most video streaming today is not served by UDP, but by TCP. Only live video uses UDP. Stored video (ex. Netflix) is all sent via TCP.
Re:Is it just me or is networking really hard? by nacs on Sunday September 13, 2015 @06:23PM (#10212982)
> Only live video uses UDP

Not necessarily. One of the largest live video streaming services, Twitch.tv, uses TCP for streaming.

Re:Is it just me or is networking really hard? by gafferongames on Sunday September 13, 2015 @09:58PM (#10213386)
Twitch is not interactive so it can just buffer for a few seconds more and handle the typical worst case of TCP. It'd probably still be better over UDP but I'm not qualified to say that for sure. I'm sure the twitch.tv guys could chime in on their choice and inform us of the pros and cons for their situation. The correct choice is application specific.
Re:Is it just me or is networking really hard? by perlgeek on Sunday September 13, 2015 @04:15PM (#10212666)
> UDP is used when it's designed by network engineers.

UDP is used when low latency and low jitter is more important than complete information.

For telefony it's better to lose a few packets than to wait a second or two for a retransmit. Some for game state, I believe.

And I do believe that TCP was also designed by network engineers :-)

Re:Is it just me or is networking really hard? by Retric on Sunday September 13, 2015 @04:30PM (#10212706)
I really depends on the game, if your writing a chess game then TCP is a much better choice than UDP. Though for anything really complex your probably going to want to use both.
Re:Is it just me or is networking really hard? by jorangreef on Monday September 14, 2015 @05:40AM (#10214276)
UDP is a lower level protocol than TCP so it's not an either/or as in "use UDP when low latency etc." is important and TCP when not.

Rather, you'd want to use a protocol built on UDP whenever you want something faster, better, more reliable than TCP.

Re:Is it just me or is networking really hard? by __d on Sunday September 13, 2015 @06:16PM (#10212972)
UDP multicast is widely used by financial trading networks to distribute state-of-the-market updates. Getting those networks to work reliably (ie. right groups distributed to right places with minimal packet loss, reordering, latency and jitter) is orders of magnitude more difficult than getting unicast traffic working.

I don't think it's going to happen. Our current networks can't even do unicast well (eg. bufferbloat, general ISP randomness).

But I'd be keen to read the OP's rants if someone tried ...

Re:Is it just me or is networking really hard? by jwatte on Sunday September 13, 2015 @07:45PM (#10213150)
Multicast at global scales is fundamentally broken. Every router on the internet would have to know about every potential peer wanting to join a multicast group.
Re:Is it just me or is networking really hard? by fanf2 on Monday September 14, 2015 @05:35PM (#10217659)
Wide-area multicast is usually one-way, which is much easier: each router only needs to know each which streams are to be fanned out to which of its nearest downstream neighbours, and where to forward upstream multicast group join requests.
[deleted]
Re:Is it just me or is networking really hard? by samplonius on Sunday September 13, 2015 @04:02PM (#10212621)
I debate whether multicast has any use case in gaming.

When online gaming started, it was fairly common to send the entire world state to all clients. Multicast helps when you are sending the same data to everyone. but online gaming started, worlds were small (think Quake I level). However, this type of model does not scale as the world size increases, and eventually the world is too big to send everything to every client.

Plus, cheating. Sending the entire world state to every client, opens up cheating vectors. The trend has been for the game server to send only state that the particular client can actually see or needs. This limits the damage of what a compromised client can do. Plus, it allows the world size to scale up.

Re:Is it just me or is networking really hard? by derefr on Sunday September 13, 2015 @04:34PM (#10212716)
Don't picture a single multicast connection socket for all clients. Picture a multicast pub/sub topic model: each client would have a regular unicast socket to send to the server, but to receive, each client would subscribe to a set of server-multicast channels, one listener-socket per channel.

For each event, the server would find (or create) a multicast-socket channel that has only the correct subscribers, and push the message once over that multicast socket. The network would then do the job of making that message arrive on every client.

And, of course, you can separately encrypt each multicast channel[1], handing out keys over a regular 1:1 TCP+TLS "control" socket to the clients. (This would usually be merged with the client's unicast "send" socket.)

Guess what design I have just coincidentally described? Cable set-top-box pay-per-view! (The original kind, not the "over-the-top" access-on-demand kind which is effectively equivalent to Netflix.)

In legacy STB PPV, each movie stream chunk is an "event" as described above; each stream is separately encrypted; and each set-top-box gets a low-bandwidth TCP-like duplex control channel to the head office to request and receive keys for streams. People requesting the same movie at the same time get put into a queue and then bucketed on ~15min intervals; each bucket gets temporarily allocated a UHF band; and then a stream is broadcast over that band to everywhere that head-office reaches, injected on the line similarly to a local public-access cable TV channel, but only existing for two hours.

---

[1] If your clients are okay with their bandwidth being wasted, you can be a bit sloppier and get away with fewer (or even just one) multicast socket(s). Imagine a multiplayer game like Starcraft: you could push every player's update events to all players... encrypted with keys in a keybag whose owners are the people the event should be visible to. Clearing fog-of-war would literally involve the client performing an action that the server responds to by sending a key to decrypt previously-received opaque event data.

This matches the model of how, say, collaboration software treats group ACLs: being granted access to a new group means being given the key to decrypt the object-change-events within the global event stream that were relevant to that group.

Re:Is it just me or is networking really hard? by kbenson on Sunday September 13, 2015 @04:31PM (#10212707)
There's nothing preventing you from sharding the world state into multiple multicast streams which clients could access/subscribe to as needed. You would still get the benefit of not sending massively duplicated data, and additionally clients wouldn't be downloading large portions of world state that are either useless, an exploit vector (even if it just exploits the game), or both.
Re:Is it just me or is networking really hard? by Dylan16807 on Sunday September 13, 2015 @04:20PM (#10212683)
Sending a fraction of the entire world state is still enormous for most non-FPS game types. It would benefit from multicast.

Also, map-hack-style cheating will always be common because of games that take the efficient rout, sending only inputs.

Re:Is it just me or is networking really hard? by Dylan16807 on Sunday September 13, 2015 @04:14PM (#10212660)
It reduces everyone's traffic. Anyone using multiple links can now install fewer links.
Re:Is it just me or is networking really hard? by aavegmittal on Sunday September 13, 2015 @09:58PM (#10213387)
Totally agreed. TCP is a great general purpose protocol which has been working "fine" in all cases. If it doesn't work well in a particular situation, then I believe its our fault that we are trying to fit it there just because it works.

Here is an example of what a custom protocol built over UDP taking in consideration the very specific use case of mobile devices accessing wireless networks, could do to achieve better throughputs than TCP, details here:

https://packetzoom.com/blog/lessons-learned-tcp-slow-start-d...

Re:Is it just me or is networking really hard? by themeisterman on Monday September 14, 2015 @11:37AM (#10215583)
Parallel TCP connection is the fasted available today. Much faster and more reliable than UDP. Cleo Jetsonic, specifically. https://www.cleo.com/products/secure-data-integration/cleo-j...
Re:Is it just me or is networking really hard? by KaiserPro on Sunday September 13, 2015 @03:21PM (#10212476)
Indeed, TCP appears to be the universal network hammer...