News for hackers. Stuff that matters.
Re:Processing billions of events in real time by mgaunard on Tuesday November 16, 2021 @04:38AM (#29238652)
Pretty much any L3 switch supports UDP multicast. What can be more rare is PIM support which is a router feature.

If anything in HFT environments, people use L2 switches for latency reasons. Those operate at the Ethernet level, so they don't really care about IP at all.

Anyway I don't see why that's specific to electronic trading or even just a low-latency concern. Sending the same traffic to hundreds of people with unicast means using hundreds of times the bandwidth, which is a huge problem.

Re:Processing billions of events in real time by posnet on Tuesday November 16, 2021 @08:46AM (#29240474)
Oh I'm well aware, I've loaded more than enough Metamako alpha firmware releases for one career.

But there are a lot of professional software devs with zero real networking experience. Sure, they may understand the text book definition of TCP or they may even have seen pictures of fibre with labels like 'this is how far light travels in a nanosecond'. But would have no idea how to calculate the serialization latency of a 10G link, let alone know the duty cycle required to saturate one.

But none of that matters in the cloud (which is where twitter is stacking their jenga tower in the original blog post). And even if both Google and AWS have custom silicon (or at least FPGAs) doing hardware offload for their internal SDN encapsulation protocols at the server level and that their custom switches all support it, it doesn't matter. They hide all of that from you the customer and rarely even acknowledge it's existence.

Re:Processing billions of events in real time by jeffbee on Tuesday November 16, 2021 @08:52AM (#29240527)
One does wish the cloud network were a little more exposed to the tenant so we can do fancy stuff. Amazon's EFA is as close as they get. On the other hand I suspect Google of having bog-standard merchant silicon switches and maybe custom silicon NICs in their latest and greatest machines but for the most part off-the-shelf stuff at the host as well.