I've used vast.ai to do some fine-tuning just a few days ago. It is indeed pretty great, though some servers fail to start up properly, or have some weird performance issues. I also wish they had more templates to try.

Yeah it works pretty well for the price - just need to be comfortable with running code and putting data on random peoples' computers (which I am for certain things). Someone on HN posted a script or snippet of output on mass-testing vast.ai servers for connectivity and configuration, and auto-labeling them using their API. Wish I could find it now... maybe with the search?

There are all these 8x 4090 machines on Vast.ai running in ASRock epyc servers and I just want to know where the hell all those are coming from. Like I want to see pictures of these setups, since there are no off-the-shelf 4090s with blower cooler setups and watercooling that many cards together is a lot of custom hardware. And the backstories, because the fact they are 4090s and not datacenter cards, are these hobbyists just building octo-gpu $18k EPYC rigs for fun? (I even saw one with 9x 4090s! gotta use up those occulink PCIe lanes) It's not ex-mining hardware since the 4090 landed after the Eth proof-of-stake-changeover.

I've been looking for an answer to this every time I check out the current vast.ai console.

There were some posts recently about 4090s being mass imported into China and the chips being desoldered/converted to lower height reference boards and blower fans.