This is very interesting, thanks for sharing. Pretty nice writeup as well. I've been running data centers full of over 100k+ GPUs for the last few years.

I can't imagine involving k8s in that though. It just seems like a huge amount of complexity that isn't really necessary. But maybe that is just me... I really like keeping things as simple as possible.

Anyway, maybe I'll have to give it a deeper look... I'm sure I can learn something from it. =)

Today, businesses and developers are keen to use cloud for deep learning. Especially with the GPU cloud instances, you pay as you go. It is much more cost-efficient comparing to having an expensive metal machine in the office. But let's switch the role now. Say you are the GPU cloud provider, and you provide the GPU environment for hosting other users applications. The problem now becomes, how can you, as this platform provider, lower down the GPU costs to maximize the profit?

coolcool