2. The point is to find some bytes that are constant for a given logical stream of related packets. Taking bytes outside of the header means taking bytes from the payload, that is by definition not deterministic. That's why everything identifies flows using the IPs + ports + protocol.
Okay, but my point there was saying "Google has millions of servers" isn't relevant, we're not looking at the entire company.
Even with a few addresses per VM, how many racks do you need to put into the same shared-compute mass? One data center is the upper limit, but it doesn't have to be the entire data center.
- let’s put 500 vms on a single one. with 128c256t CPUs it’s easy
- say you can fit 30 of those in a single rack (the common rack is 42U) due to power constraints
- and place 10 of those racks
That’s 500 x 30 x 10 = 150000 nodes to address. With 10 racks you already blow past the scaling limits of the common datacenter switch when it comes to MAC addresses. Here are the limits for Cisco’s Nexus 9000 series, a very common datacenter switch: https://www.cisco.com/c/en/us/td/docs/switches/datacenter/ne...
I've seen computers at moderately sized LAN parties (talking 100 nodes, far from the large or even massive events) that were literally crippled by the broadcast traffic. At some point the flooding and layer 2 discovery (ARP) would do the same as well.
Limiting the broadcast domains with layer 3 really makes the Internet possible. Sure, you can have less overhead and simply do layer 2 only, and really it is completely possible. It's just such a rare use case that it in practice isn't important enough to actually do.
2. Use some bytes outside of the header?
3. I get the impression this needs hardware to really use well anyway.