As a company we operate a server with consumer cards. Running your own consumer Nvidia cards is very cheap compared to buying time on servers with enterprise Nvidia cards. On the cloud you are paying a premium for being on the cloud, and on top of that the premium for having datacenter-licenced cards.

Of course this calculation depends on how bursty your compute requirements are and how much you pay for electricity (datacenter cards are more power efficient)

Isn't the proximity of your compute to your data a huge consideration too? I imagine that the egress data costs (cloud storage to your local GPU) could grow quite significantly if you're crunching enough data.

I guess if you're rolling your own compute clusters, you've probably rolled your own storage solution too?

I work in machine learning and bought an Nvidia card for my PC with the intent of doing ML on some personal projects. But I bought a 2070S, and would never drop that much on a 3090 for personal use.

So yeah, the huge gap ML capabilities between AMD and Nvidia are a selling point, but probably for a small enough group that it doesn't make a difference.

I went for the 3080 for CUDA which isn't entirely for ML but also things like Blender, video rendering (plus 4K RTX Minecraft, of course), etc. If I hadn't had to upgrade my PC the month before, I'd probably have gone for a 3090 for extra future proofing.

I am doing this right now.

Using cloud services for work would mean using the exorbitantly expensive services from Azure, to say nothing of the painful and annoying experience that using Azure is at the best of times.

Instead, I can spend a moderate amount, build a machine that is more than adequate for work requirements will last ages, write it off on tax and still end up spending way less than I would have renting a cloud machine for a couple of months.

Yes, many people do independent research into ML and prefer to develop with local resources.