I use Apple and AMD chips for desktop inference. nVidia is still cheaper for anything beyond that.
I really only have it for data locality - some customers simply don’t have approval for me to share data from their customers with random cloud inference providers. I also use Bedrock for that but it’s expensive
Free markets can be abused. Demand, in NVIDIA's case, seems to be not all that Jensen, and the company, had been stating. If the core demand was AI companies that were being funded by NVIDIA itself - then it wasn't organic demand.
If "... at least $200 billion worth of NVIDIA AI GPUs are sitting idle in warehouses after being purchased by cloud giants that lacked either the electrical grid capacity or immediate end-user demand to power them on" [0] is true then the demand wasn't there. There's zero reason to pre-purchase a GPU a year, even 6 months, ahead of build. Sure, maybe Google / Amazon / SpaceX buy during build - but these companies actually build their DCs, they don't buy GPUs and store them in a warehouse for quarters on end unless something went wrong (maybe like in Finland [1]). Jensen may very well be in a lot deeper than anyone really knows. The reporting as of late has stated that commitments by NVIDIA to memory suppliers was coming in and there was likely not enough non-circular capital to cover. The reality is Jensen likely thought he was going to be the AI kingpin, having a piece of all the AI pie and that he'd have enough time to recoup his ponzi-like investments in the downstream by paying for his own products.
I, personally, don't think we've even seen the start of this story. It's probably a huge mess starting with Jensen and flowing down through his kids. I can't imagine everything was on the up and up given the scale of dollars involved. It almost feels like the affair was a scapegoat to get out of the hot seat. Ultimately time will tell.
But, TL;DR - if you think the demand for NVIDIA was completely organic, well then I have a crate of Founder's Edition 5090's to sell you for dirt cheap.
[0] https://wccftech.com/jensen-huang-financed-ai-startups-absor... [1] https://www.reuters.com/business/environment/finland-launche...
People I know with adequate space and power have also heard these rumors and have looked high and low for parties with these billions worth of idle GPUs-- either to purchase their hardware or offer them hosting-- and cannot find them.
I think people are just making this up. If I'm wrong and you know someone in need, let me know and I can connect them with someone with 2GW+ of capacity.
Sure, it sounds simple but again, I don't think we understand any of the complex web of moving parts that are involved here.
So, you will not have money and you are going into a field with big competition where all you have to go on is "well, I surely could do better." Not impossible, but that takes existing establishment in the field. You are not going to see mom and pop chip companies. Not without some major breakthrough that sets them apart.
This is the free market; demand is high and supply is limited so price goes up. Don't like it? Start making your own and benefit from the higher prices!