Isn't the proximity of your compute to your data a huge consideration too? I imagine that the egress data costs (cloud storage to your local GPU) could grow quite significantly if you're crunching enough data.

I guess if you're rolling your own compute clusters, you've probably rolled your own storage solution too?