All of the major vendors MLAG implementations are limited to two agg switches, assuming you have a pair of 24 ports and a 48 port rack level switch hanging off each, that still puts total servers at less than 1000.
What stops someone from having multiple zones within a datacenter interconnected with suitably large LAGs thereby avoiding being limited by the port capacity of any one device?
Why is the exact number of servers important to the topic, or is it just to satisfy your curiosity? :)
> There are maybe, what, a thousand people in the U.S. that have worked on network administration for something GitHub's size?
I didn't intend to deflate any nerd egos, just putting things in realistic terms. GitHub is cool. It's the biggest in its industry. It's definitely an interesting problem. But it's still hosted within Rackspace, which in itself isn't that big.
One big recommendation I'd offer is to have a secondary network with as few moving pieces as possible (think a dumb unmanaged Netgear switch or two) to run things like heartbeats and DRBD over. That sort of thing should not be living on the same segment as your production traffic.