When you say you are not just doing inference, you mean you are also training your own llms? I am curious what other things can be done.
Fine tuning, and yeah training my own, experimenting with architectures and learning how it all works. Been a lot of fun
Re:Was my $48K GPU server worth it?
by nostrebored on Thursday May 21, 2026 @09:40PM
(#48230966)
$6500 AUD can get you a good chunk of B200 time on any of the GPU neoclouds :)
Less than I expected, though! And I get to run this all through the night
I do still use Vast and Runpod for things too, but it’s much nicer to test a fine tuning run here to make sure I’m in the ballpark
I also did literally say “It's not financially a good idea, renting is better than owning” so I’m confused why I have two people telling me that
Also it’s just far more fun to play with something tangible to me :)
You could just rent a bare metal server with those specs
Yes I could, but that is annoying because of spot pricing and having my instance shut down, and it has fluctuating prices
It’s also annoying because then I need to make sure my little “lab” setup is well automated, and I’m lazy :)
Also, I literally said “ It's not financially a good idea” so I’m confused why you think I don’t know that.
Spot pricing and instance availability don’t apply to on metal hosting. You’d have your own machine dedicated to your own use only, at a locked in price.
> renting really does beat owning, and cloud beats both
Wait, I went and looked far and wide today and I can’t find anything with ~100GB of VRAM that isn’t $20k a year, what am I missing
That's too small. I was quoted a machine with ~1.5TB of vram, for $10k/mo. This was the minimum node size in the AI data center I was talking with -- they don't make smaller nodes that you can lease as bare metal. There is no public pricing though, and you had to know people to get in.
It's not financially a good idea: renting really does beat owning, and cloud beats both if you're only running inference on these machines. But I'm not just doing inference, and as a thing I can do silly stuff on to learn, it's hard to beat!