That said, 80G for something that you picked up for cheap could be an absolute steal these days.
Currently listening to the soothing drone of my $200 Octominer cooling my cards while I'm fine tuning a model (almost makes a good space heater).
https://www.reddit.com/r/LocalLLaMA/comments/1wjr59t/768gb_v...
How are you dealing with the RAM limit (32GB)? I had upgraded mine to an i7 but it was unstable so I’m back to the stock Celeron class CPU.
Bad RAM isn't a huge issue as you can flag and ignore the relevant regions and be down a few MB.
I even looked into blog posts of using them or unlocking them somehow but I never thought to think that they actually had the full HBM memory and all just fused away??? wow
After which I sold most of them for $2000-$2600 (I believed the panic about the new mac studio launch) instead of holding them to sell for like $5k and making my annual income in a week. And also I failed to do the capacitor mod after buying a couple hundred in soldering equipment. But at least I sold most of them for monero, and that's gone up since and allowed me to avoid massive ebay fees.
The moral of the story here is that you can fantisize about getting in at the bottom, but even if you did you may not have rode it to the top. Plenty of people bought bitcoin when it was half a cent and sold at the local maxima to net $30k and spent it on a used corrolla.
I think my story may make your grapes less sour.
Edit: I knew at the time of purchase that the 10GB->80GB unlocks were problematic but the 8GB->64GB cards were solid. The pricing reflects this, so thats just skill issue.
As for v100, p100 etc I considered them, but they make zero sense as they don't have int8/int4. Everything gets upscaled in them which slows the inference down.
Anyways, that's what I tell myself as I taste my bowl of sour grapes.