News for hackers. Stuff that matters.
Re:Large-scale cluster management at Google with Borg by nrr on Thursday May 07, 2015 @04:29PM (#9508058)
Even with high-performance networks in place, you absolutely must budget your bandwidth between management communications, low-latency service communications, and batch job communications, amongst possibly even others. Having data locality is still a really, really big priority.

(Furthermore, building out separate networks is extremely cost-prohibitive once you've expanded out to a total footprint in the low-1e6 ballpark.)

Think about it this way: In recent history, we've added L3 cache to commodity processors because keeping data on-chip is still faster, in 2015, than going off-chip to RAM. The same analog holds when trying to fetch from local disk versus remote disk.

Brendan Gregg actually provides a really good time-stretched view of various operations in his book on performance tuning. Go give that book a read; there's some _really_ good stuff there.

There are no comments on this comment yet.