In my experience doing detailed projections of exactly something like this - with racks full of GPU compute power - the infrastructure has paid itself after one quarter, maybe two, depending on what volume you're at. There very rarely is a reason to use GPU compute on the cloud - and advantages start as quickly as with just one single GPU.

I feel that a lot of posts like this might be under-representing the true costs of running your own hardware. I was only tangentially associated with a large-ish operation and I can tell you that there are loads of things that take a lot of time but are often over looked. Things like detecting and replacing bad hardware. HDs don't last forever and when they go bad it's not fun; especially if you have to source some specific model for your 7 year old server platform.

Understanding your licensing and warranties is another huge cost that people don't take into account. We used to spend hours and hours figuring out if we could replace systems and what it would cost us.

Finally, you have to dispose of all that hardware when it gets too old or out of warranty. If you've never had to do that you probably have no idea how hard it is to do it correctly and so that it satisfies your SOC2 auditor.

All of these things (plus more) really add up. It's not just purchasing the components and installing them in racks. The management is probably even more expensive than the hardware.

And all of these problems go away with a cloud provider.

Your VAR should be doing a lot of that work for you - sure you need to understand licensing but if you have a good rep they should be providing you with up to date info on all of this.

HDD replacement is trivial and some COLOs can even do it for you - and any good vendor will have a thing setup where your appliance automatically notifies them and a new drive is overnighted to your COLO. Sure - maybe a few times a year you drive out to swap a drive.

If you do not have good monitoring setup then that is entirely fixable - there are many stellar solutions these days out there. Hardware is easier than ever.

Right, I agree that the cost might be underestimated, but I was specifically talking about the capital expenditure required to “eject” from the cloud, not the ongoing operating costs.