You're building a common infrastructure to address privacy concerns about the use of shared infrastructure??
Basically make your own hpc provider for academic purposes, and rent out extra capacity to smaller commercial interests to utilise that spare capacity and also prop up the local economy with usable research compute.
how many successful careers include 8 months in prison
The allegations haven't been proven to be true. Thats what the courts will decide on when the class action continues.
- Cutting DRAM production in 2022 as a result of low prices/demand made sense if they didn't expect the market to recover as quickly as it did. A lot of the recovery is also likely caused by the start of the AI boom.
- The deals OpenAI made with SK Hynix and Samsung was a smart move by OpenAI where they drove up prices for everybody else while securing "cheap" supply themselves. I dont think its clear whether SK Hynix and Samsung came out ahead in that deal as the demand likely would have risen either way and it wouldn't have needed to go up by as much as it has now.
- Moving production to HBM also makes perfect sense when that is what the market demands and is willing to pay a premium for.
It made less sense back when billionaires needed to get on trains and spend hours to collude. Now they just make podcast appearances.
See the prisoner's dilemma.
However even knowing everything, you still have incentive to see how much you can cheat without getting caught.
That's a matter of course. This goes way beyond our wallets. They are actively pricing us out of computing. They are deliberately pushing computers back to the big iron mainframe era.
Computers are subversive: they democratized cryptography, copyright infringement, all sorts of things. Now that humanity has developed AI, they will not cannot tolerate mere citizens having access to powerful computers any longer.
If we don't fight this somehow, it's over for us. Everything the word "hacker" ever stood for will be destroyed. We need some kind of second amendment but for computers.
(Speaking of which—I've seen people suggest a seti@home-like approach to creating a frontier-level, fully open LLM. C'mon, guys, let's vibe code this thing!)
I'm one of them! Every once in a while I ping the community about this idea.
https://news.ycombinator.com/item?id=48695813
They're no substitute for high performance computers though, especially since that's where all our powerful tools execute. We can't develop software on an MCU. We must ensure access to our work stations if we aim to maintain our digital freedom.
> The judge found that long-term HBM lock-ups, refusals of long-term conventional DRAM contracts, Micron's Crucial closure, fab delays, the failure to expand despite record prices, near-identical customer vetting and the companies' prior cartel history supported a plausible inference of conspiracy.
They didn't even announce plans to expand production until after the news of this class action lawsuit broke, despite massive DRAM price increases.
“To date, five manufacturers have pleaded guilty to their involvement in an international price-fixing conspiracy between July 1, 1998, and June 15, 2002, including Hynix, Infineon, Micron Technology, Samsung, and Elpida.[3]“
What hasn't happened is that they have been found guilty in the latest class action lawsuit. Which is what this discussion is about.
So far the court has only decided that the lawsuit can proceed, which indicates there is some substance to it.
In my opinion, the fact that this has happened before, and that the companies involved eventually plead guilty, is an important piece of information for people following this story. Especially in light of the statements from the companies involved today.
It is the complete opposite of the current situation.
Just the cost of doing business.
Work for however long, shut down. I do this with linode and their GPU VMs. If you shutdown nightly and on weekends, the costs drop dramatically on a monthly spend.
I only need it a few hours per day, so very cheap for custom compute.
The custom volume keeps your persistent install, configs, work in progress, etc.
You can programmatically kick off a linode, attach the volume, then boot, as well as delete, all via curl or python.
So not variable rates, but no rates when the beast is deleted.
The project ran out of budget roughly three months in. From what I heard, status meetings consist of everyone getting together once a month, asking "have we gotten any more budget yet? no?", and then wait until the project runs out.
Cloud providers need to make a profit too, and they've gotten very good at ensuring that their clients forget this point. The whole advantage of cloud compute is that you don't pay for the time you're not using it, but once you reach a certain point you're the one covering the overhead for everybody else.
The glut of AI chips will lower prices meaning consumer GPUs start looking high margin. And if they pile into it again, prices will come down.
Kevin O'Leary's 20 GW data centre is never getting built. Maybe he'll do a "phase 1" at a few hundred MW, but he's not going to build the largest power station in the world.
What you probably want instead is a Mac Studio with an M5 Ultra. The 256GB model is ~$11k. The problem? Demand is so high that the lead time last I checked was March. But this will give you 256GB of shared memory, well above the 32GB you'll get on a 5090.
If you can live with 96GB (or less) on a Mac Studio it's quite a lot cheaper and the lead times are far quicker.
Well not really, there is the AMD Radeon AI PRO 9700, which is a RX 9070 (or possibly the XT, dunno) with 32GB VRAM. It does use slower GDDR6, but it has "held its price better" (for some definition of better) and is still available for like 2.3k€ (this includes taxes). Though I remember when I was thinking of buying one when it was 1.4k not too long ago and didn't :( not I just don't bother looking anymore.
The RX 7900 XTX has slightly higher bandwidth at almost 1kGBps compared to AI PROs almost 650GBps but has only 24GB instead of 32GB, though I can't find the card on sale anywhere rn.
In the end I see it mostly as a "is the massive jump in price worth it, or do you risk it and maybe get multiple AI 9700s instead and deal with ROCm instead". I can't tell how good ROCm has gotten in regards to training and such, but for inference Vulkan on my 9070 XT is pretty solid.
A Mac studio won't be able to keep up with my rig.
The market has gotten so tight that antiquated hardware has become competitive. Too many of us are trying to buy the same aging platforms for exactly this reason.
Finally secure a research grant and had budgeted money for a new compute workstation and the RAM prices are killing my budget, because I'd love to get at least 256 GB of RAM and that's 4-5 grand now. Ridiculous, especially since I'd like to have a GPU in it with at least 32Gb of VRAM for medical image model training, so that is also like 4-5 grand too.
Do I wait two years and hope, or pay, isthe question I keep asking myself.