There is virtually not a single "enterprise" grade product which can't be made at least 50% cheaper (or sometimes 10 times...) with off the shelf consumer grade hacked hardware....
Enterprise products always have a pretty steep markup, but what you lose with those 1200$ GPUs is both features (e.g. virtualization, thin provisioning, DMA/Cuda Direct etc.) and support. When you buy a 5000$ CPU over a 500$ with the same performance what you pay for is reliability and support, if you don't care about that then fine, but when you need to launch a 100M$ service on top of that platform you won't really care about the price tag it's all in the cost of doing business.
DMA/CUDA Direct? Say Hello to P2P and staged MPI transfers, faster, cheaper (and usually better). Know your PCIE tree FTW.
Support? As someone who has been playing with GPUs for over a decade, bugs get fixed in the next CUDA release no matter Tesla or GeForce, if ever.
$100M service? Yep I'm with you. But I prefer a world without a huge barrier to entry to building that service, especially a barrier built 99% on marketecture. I want to build on commodity hardware and deploy in the datacenter.
Unfortunately, sales types seem to hate that outlook.
Edit: "replace" is probably not the right word, this is work that the old systems don't do well, but they aren't throwing out x86 racks for gpus of course. It's just instead of buying more of the same for machine learning applications.
If nothing else is that because you cannot virtualize Geforce line GPU's, there is no CUDA Direct or NVLINK support etc.
If you are telling me that Google is buying Geforce GPU's and flashing the bios with a custom bios ripped off a Quadro card so they can do PCIe passthrough in a hypervisor and initialize the cards then sorry not buying it.
They use containers to isolate and throttle different tasks/jobs running on the same hardware.
At their scale, virtualization would be significantly wasteful in terms of manageability and overhead.
I'm not talking about running virtual OS, I'm talking about things like rCUDA, GPU direct and RDMA.
But still even for their containers solution they need support for gpu passtrough and vGPU if not they can't run containers.
NVIDIA doesn't allow you to run GeForce cards over a hypervisor.
source?
perf/W is a useful number if you are trying to stack a data center full of these things and electricity (incl for cooling) is essentially your only cost. But if a car is using 300Wh/mile or, in an ICE, generating 100kW in excess heat it is an entirely pointless metric.
(Just for clarification: no one is using the Drive PX2 in a data center. Just look at these connectors:
http://images.anandtech.com/doci/9903/NVIDIA_DRIVE-PX-2.jpg )
But in the meantime, where we haven't actually solved the problem even when the trunk is full of racks with high-end boxes, it seems silly to mandate that, in the classic GM "compromises safety" sense.
The powertrain uses 300+ Watts, so it seems like a manageable number if it is correct.
Anandtech reported [1] that the TDP of the whole board is around 250W. The Tegra SoCs are probably around 10W, but that doesn't get you much in the way of GPU horsepower.
1: http://www.anandtech.com/show/9903/nvidia-announces-drive-px...
Point stands though.
The point I made missed the point though, it apparently isn't the energy consumption they are concerned with, other people in the thread are pointing to limiting/getting rid of the heat being the main concern.
The components of this system need to generate next to zero heat and they need to be placed in all sorts of inconvenient locations. That's causing automakers to desire extremely low-power dedicated circuitry over GPUs.
Consider as an example the C7 Corvette: despite its enormous blindspot, it still doesn't have blindspot indicators because they can't find any way to cram existing sensors into the thing. There are more examples, that's just the one with which I'm most familiar because I balked on purchasing one over this despite an insanely great deal on it at the time.
And they may continue to win because successful execution of an 80% product is worth far more than a 90+% powerpoint processor cough TenSilica et al. cough, or because this is such a huge potential market, it might actually go to a successful competitor. 2018 and beyond will be very interesting.
For while it's really desirable have the deep learning equivalent of x86 assembly language (CUDA) across a full stack from training to inference, in the end, IMO cost will be king. I'm not a big fan of $150K high-end servers filled with $5000 GPUs that can be bested with clever code on a $25K server fill with $1200 consumer GPUs. But I am a huge fan of charging what you can while you are unopposed. It's just that I think that state is temporary.