These aren't rack mount products aimed at cloud providers, they are essentially mini workstations. What are you calling "at scale" for this? You're basically always pairing one of these machines to one physical person sitting at it whose time is being paid for as well (even for a solo creator, time is still money). It's a terrible tradeoff to save pennies per hour on power to turn around and pay the person dollars more per hour waiting on results.

That seems like a really bad way to spend money. Why limit a person to a single workstation if the workstation is the limiting factor? This is where we get clouds or if it must be done locally, rack mounted systems with many cards.

If you are doing it solo, with "just the hardware you happen to have", it matters a bit less. If you are doing it constantly to make money, and you need a lot of power, buying a one-person desk-machine makes no sense.

The cost of powering my 3090 for a year is now more than the cost (RRP) of a 3090.

Where do you live that power is anywhere close to that expensive? And are you overclocking your 3090?

Even assuming literal 24hr/day usage at a higher, "factory overclocked" 450w sustained, at a fairly high $0.30/kWh that's $1200/yr. Less than half the retail price of a 3090. And you can easily drop the power limit slider on a 3090 to take it down to 300-350w, likely without significantly impacting your workload performance. Not to mention in most countries the power cost is much less than $0.30/kWh.

At a more "realistic" 8 hours a day with local power pricing I'd have to run my 3090 for nearly 10 years just to reach the $2000 upgrade price an M1 Ultra costs over the base model M1 Max.

UK electricity is $0.28/kWh but will be $0.36/kWh from the end of the month - my business is quoted at $0.60/kWh fixed for the next 12 months.

At $0.36/kWh - card alone @450W ~running cost ~= RRP 3090 $1,499

Yes, can power it down to be more efficient, however, that effectively agrees with the previous comment that PPW matters.

Wholesale electricity price in some EU states is €550/MWh today. Most EU states are above €250/MWh.

prices in Europe right now are considerably higher than $0.30/kWh

It seems safe to say prices right now are not the norm due to, you know, that whole war thing going on that is impacting one of the EU's primary power supplies. The 2021 EU average was otherwise $0.22/kWh.

yeah but let's how fast we have those "normal" prices again, maybe this is the new normal, who knows.

The war is a factor going forward, but we got notified of the price increases before the war started, they relate more to the piss poor planning of our rulers than any external factors. Unless the people in charge are going to suddenly start planning for the future this will be the new normal.

But do you really run it all year round?