News for hackers. Stuff that matters.
Re:AMD Launches Instinct MI300X AI Accelerator, Up to 60% Fa... by filterfiber on Wednesday December 06, 2023 @03:01PM (#38548752)
I really hope AMD gains more ground on the software side.

I know they are "technically supported" by onyx/pytorch etc. But almost every project I've seen there's a bunch of asterisks, a bunch of people needing different work arounds, and subpar performance compared to the hardware.

Maybe it's because most people have their consumer cards? Nvidia however doesn't have this issue, people on pascal are only just now having compatibility issues with not supporting a high enough cuda version.

The dream is they release a 4090-type competitor for a few grand, it doesn't even need full 4090-level performance, just strap 48gb+ of HBM and it would be a absolutely incredible deal for individuals learning/researching -> who then use the full enterprise stuff for full training/commercial inference.

Re:AMD Launches Instinct MI300X AI Accelerator, Up to 60% Fa... by sbininit on Wednesday December 06, 2023 @05:28PM (#38550372)
I suspect that those who would otherwise be buying thousands of H100s would simply put in the dev work to make the software function well on AMD hardware
Re:AMD Launches Instinct MI300X AI Accelerator, Up to 60% Fa... by idonotknowwhy on Wednesday December 06, 2023 @05:34PM (#38550424)
I'm hoping for the same thing, but with Intel ARC. Intel gpus are cheaper, but have even worse ML software for consumer projects
Re:AMD Launches Instinct MI300X AI Accelerator, Up to 60% Fa... by frognumber on Wednesday December 06, 2023 @06:41PM (#38551010)
I'm optimistic.

* Intel has a history of getting this type of software working.

* AMD doesn't.

I don't think I'll ever trust AMD after... experiences, but Intel I'm bullish on Intel.

Arc A770 seems like by far the bang for the buck once software lands. 16GB for $300 if you find a good deal. Decent performance. Stick four of them in a system, and you've got 64GB for $1200 and a pretty decent amount of compute. Buy five systems like that, and you can run the largest open-source LLMs out there with Alpa for ~$6K.

That brings Facebook's GPT3-grade OPT models from corporate down to serious enthusiast level.

Re:AMD Launches Instinct MI300X AI Accelerator, Up to 60% Fa... by rreichman on Wednesday December 06, 2023 @03:28PM (#38549055)
Their software is definitely improving and will get a whole lot better in the next few years. It's worth many billions for companies to exit the Nvidia lock-in.
Re:AMD Launches Instinct MI300X AI Accelerator, Up to 60% Fa... by fariszr on Wednesday December 06, 2023 @04:34PM (#38549809)
As long, as it doesn't support normal consumer hardware, AMD won't get the support Nvidia GPUs enjoy from community projects.
Re:AMD Launches Instinct MI300X AI Accelerator, Up to 60% Fa... by rreichman on Wednesday December 06, 2023 @03:28PM (#38549046)
According to this and SemiAnalysis[1], the MI300X seems impressive. Better than Nvidia's H100 and maybe a bit inferior to the H200. Why is Nvidia worth $1 trillion more?

[1] - https://www.semianalysis.com/p/amd-mi300-performance-faster-...

Re:AMD Launches Instinct MI300X AI Accelerator, Up to 60% Fa... by tedunangst on Wednesday December 06, 2023 @05:01PM (#38550138)
$9B in profit vs $300M will have that effect.
Re:AMD Launches Instinct MI300X AI Accelerator, Up to 60% Fa... by fvv on Wednesday December 06, 2023 @07:11PM (#38551232)
Xilinx amortization..
Re:AMD Launches Instinct MI300X AI Accelerator, Up to 60% Fa... by rreichman on Wednesday December 06, 2023 @05:20PM (#38550300)
Sure but the future is not the past
Re:AMD Launches Instinct MI300X AI Accelerator, Up to 60% Fa... by riku_iki on Wednesday December 06, 2023 @05:44PM (#38550523)
AMD p/e is 900 vs 60 for NVDA, so investors already heavily invested into AMD future.
Re:AMD Launches Instinct MI300X AI Accelerator, Up to 60% Fa... by tedunangst on Wednesday December 06, 2023 @06:27PM (#38550899)
Time to buy those calls.
Re:AMD Launches Instinct MI300X AI Accelerator, Up to 60% Fa... by mattnewton on Wednesday December 06, 2023 @03:39PM (#38549193)
CUDA
Re:AMD Launches Instinct MI300X AI Accelerator, Up to 60% Fa... by rreichman on Wednesday December 06, 2023 @05:21PM (#38550305)
CUDA is becoming less important over time
Re:AMD Launches Instinct MI300X AI Accelerator, Up to 60% Fa... by frognumber on Wednesday December 06, 2023 @06:43PM (#38551030)
I'm not sure it is.

You can do more with non-CUDA now than ever before.

At the same time, the number of things you can do only with CUDA is greater than ever before too.

The whole field it growing, and it's hard to compare.

Re:AMD Launches Instinct MI300X AI Accelerator, Up to 60% Fa... by Racing0461 on Wednesday December 06, 2023 @05:35PM (#38550432)
We need competition in the ai gpu space.