If you are doing it solo, with "just the hardware you happen to have", it matters a bit less. If you are doing it constantly to make money, and you need a lot of power, buying a one-person desk-machine makes no sense.
Even assuming literal 24hr/day usage at a higher, "factory overclocked" 450w sustained, at a fairly high $0.30/kWh that's $1200/yr. Less than half the retail price of a 3090. And you can easily drop the power limit slider on a 3090 to take it down to 300-350w, likely without significantly impacting your workload performance. Not to mention in most countries the power cost is much less than $0.30/kWh.
At a more "realistic" 8 hours a day with local power pricing I'd have to run my 3090 for nearly 10 years just to reach the $2000 upgrade price an M1 Ultra costs over the base model M1 Max.
At $0.36/kWh - card alone @450W ~running cost ~= RRP 3090 $1,499
Yes, can power it down to be more efficient, however, that effectively agrees with the previous comment that PPW matters.
Watts are dollars that you'll continue spending over the system life. It matters because you can only draw so many amps per rack and there will be a point when, in order to get more capacity, you'll need to build another datacenter.
These machines are commonly used by professionals in industries like movie and music production. They don't care what the power bill is, it's insignificant compared to the cost of the employee using the hardware.
Oh... They do. At least, they should. If a similar PC costs $500 less but you spend $700 more in electricity per year because of it, at the end of the year, your profits will show the difference.
I ran a visual effects company a decade ago. We bought the fastest machines we could because saving time for production was important. The power draw was never a factor; a few catered lunches alone would dwarf the power bill.
The silly thing about it is that most of the special engines can now be flashed into an FPGA which is becoming more common in the big clouds so special offload engines aren't that big of a deal when they are missing. So in some cases you can have your cake and eat it too; massive parallel processing and specialised processing in the same server box without resorting to special tricks (as long as it's not suddenly getting blocked in future software updates).
Honestly asking because I’m kind of out of the nvidia loop at the moment.
Legally: I assume no cloud provider will assume the legal risk of telling their customers "and here you have to break the EULA of the NVIDIA driver in that way to use the service". In Europe where the legal environment is more focused on interoperability, this might not be as much of a problem, but still it may be too much risk.
There is raw performance, but there is also performance per watt, availability and scalability (which is both good and bad - M1 is available, but there is no M1 Ultra cloud available). If you want a multi-use setup, an RTX makes more sense than most other options, if you can get one and at a reasonable price. If you want a Mac, the M1U is going to give you the best GPU. In pretty much all other setups there are so many other variables it's hard to recommend anything.