News for hackers. Stuff that matters.
Re:Espresso – Google’s peering edge architecture by walrus01 on Wednesday April 05, 2017 @03:45AM (#14040451)
What I mean is that they do not run their links to saturation in the same way as an ordinary ISP. And because their traffic patterns are very different than an ordinary ISP, and much, much more geographically distributed, they can do all sorts of fun software tricks. The end result is the same: Low/no jitter and no packet loss.

As contrasted with what would happen if you had a theoretical hosting operation behind 2 x 10 Gbps transit connections to two upstreams, and tried to run both circuits at 8 to 9 Gbps outbound 24x7.

Re:Espresso – Google’s peering edge architecture by danpalmer on Wednesday April 05, 2017 @07:08AM (#14041200)
For clarity, do you mean that Google can, for example, run to 99% saturation all the time, whereas a typical ISP might have 30-40% average, with peaks to full saturation that causes high latency/packet loss when it occurs?
Re:Espresso – Google’s peering edge architecture by pas on Wednesday April 05, 2017 @12:15PM (#14043177)
Yes, that's about right. Since they control both sides of the link, they can manage the flow from higher up on the [software] stack. Basically, if the link is getting saturated, the distributed system simply throttles some requests upstream by diverting traffic from places that result in traffic over that link. (And of course this requires a very complex control plane, but doable, and with proper [secondary] controls it probably stays understandable, manageable, and doesn't go haywire when shit hits the fan.)
Re:Espresso – Google’s peering edge architecture by collinmanderson on Thursday April 06, 2017 @10:08AM (#14050603)
So I wonder if that means they can do TCP control flow without dropping packets.
Re:Espresso – Google’s peering edge architecture by pas on Tuesday April 11, 2017 @05:48AM (#14086206)
I guess they do drop packets (it's the best - easiest/cheapest/cleanest - way to backpropagate pressure - aka backpressure), but they watch for it a lot more vigorously. Also as I understand they try to separate long lived connections (between DCs) from internal short lived traffic. Different teams, different patterns, different control structures.