Re:CoreOS (YC S13) Raises $12M to Bring Kubernetes to the En...
by brazzledazzle on Tuesday April 07, 2015 @06:48AM
(#9333013)
You should look at their measurements and assumptions in context. As you can see from the URL, it was written in 2011 when the NVMe working group was first formed. It was also written in the context of cluster-based applications in a data center and specifically mentions SSD and cost effectiveness. Storage cost effectiveness is critical at these scales because your data is growing by terabytes per day.
You also mention blades, which goes into the next point of context which is that operations like Google and Facebook don't utilize blades like you might expect working at your average enterprise because they aren't leasing rack space or working with a limited amount of physical space. They don't need the same U to performance ratio so they can save money by using commodity hardware. Their applications also scale readily, so the loss of entire boxes is meaningless within a certain threshold.
The paper seems to peg local disk bandwidth at 150Mbps, and then compare it to remote network disk access at... 150Mbps. NVMe is going to grant us 2.2GBps bandwidth and 450K IOPS (from a single consumer-grade product), so that paper is off by more than an order of magnitude. Local disk is non-volatile storage sitting a PCIe lane away from your CPU. I just don't see how disk locality is not going to be crucial for many workloads, for decades to come.
In 2020 a flash-only SAN isn't going to deliver 20Gbit/sec to each of 100 blades in the rack. A 4TB NVMe card on each blade will though...
Look at Intel's latest Xeon-D SOC, yeah, it's got Dual 10GBE, but you're not going to get 7.7GB/s over that... [1]
[1] http://www.intel.com/content/www/us/en/benchmarks/server/xeo...