that's really surprising to me, because I used to run a GPU cloud that was cheaper than amazon, and the margins were insane.

I've looked for some GPU on the web and it's super expensive to rent. I was expecting to rent that for say 50$ per month (considering I can rent some webspace for less then 10$ a month) but everything was more expensive like 600$/month (I look for entry level GPU, used just a bit, I'm a student) ... Did I look correctly ?

if you want an always-on gpu for $50/mo? That's unreasonable. Consider how much "building from scratch" even a cheap "minimal" GPU unit costs (let's say 3k GPU + 1k supporting PC), whereas a cheap "building from scratch minimal webspace" is a fraction of a single CPU unit, so let's say $100-ish/unit dollars, to be really generous. A rented non-fractional GPU should be easily minimally 40x the cost of a website.

Don't expect these ratios to be authoritative; there are obviously different economies of scale and many many details I papered over, but this should just give you an idea of the back of the envelope "relative costs"; just that 5x is way out of the range you should have guessed.

It all comes down to utilization as a percentage of time. If the workloads are infrequent enough, GPU cloud providers on going to crush the onsite equipment.

However... I was also sort of surprised, because last time I priced this out, you can have pretty low utilization rates of the equipment and still come out ahead.

the other thing to consider though is that if you are under pressure by management to not utilize GPU cloud, you might do fewer experiments than if you had at least a GPU workstation under the desk. (that, presumably, you can share with other workmates)

I appreciate your point about "they can't hire the expertise they need" but I feel it's missing the "because": because those hires are so expensive? Because they only exist as consultants? Why can't they hire the expertise?

For company I am working at, it is acceptable to pay more for even long term consultanst, than to pay market price for experienced developers. The reason explained was, that it is different budget line - HR does not want to "overpay" for emplyees, so consultancy is the only option.

I would so that about 3/4 places I've consulted, I wouldn't want to work full time.

Mostly because they simply didn't know how to run an IT office, and so they couldn't build a culture where good IT people want to work. When they finally get a strong candidate, they end up overloading the poor hire, because finally, it looks like they can get their infrastructure fixed or dream project off the ground.

because nobody wants to be hired for the position. it isn't even about the money, there's simply no candidates - and any candidates competent enough are already consultants and/or freelancers.

I.e. those hires are expensive. I am sure there would be candidates with an offer of $1M/year.

of course, everyone has a number - but also, for that money you'll just hire a consultant and will probably come out ahead.