News for hackers. Stuff that matters.

A comment on Testing AMD's Giant MI300X. Parent comment

Re:Testing AMD's Giant MI300X by kd913 on Tuesday June 25, 2024 @12:24PM (#40790430)
You do know that Microsoft, Oracle, Meta are all in on this right?

Heck I think it is being used to run ChatGPT 3.5 and 4 services.

[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by softfalcon on Tuesday June 25, 2024 @12:30PM (#40790511)
I feel like people forget that AMD has huge contracts with Microsoft, Valve, Sony, etc to design consoles at scale. It's an invisible provider as most folks don't even realize their Xbox and their Playstation are both AMD.

When you're providing fab designs at that scale, it makes a lot more sense to folks that companies would be willing to try a more affordable option to nVidia hardware.

My bet is that AMD figures out a service-able solution for some (not all) workloads that isn't ground breaking, but affordable to the clients that want an alternative. That's usually how this goes for AMD in my experience.

[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by sangnoir on Tuesday June 25, 2024 @02:39PM (#40791951)
If you read/listen to the Stratechary interview wirh Lisa Hsu, she spelled out being open ro customizing AMD hardware to meet partner's needs. So if Microsoft needs more memory bandwidth and less compute, AMD will build something just for them based on what they have now. If Meta wants 10% less power consumption (and cooling) for a 5% hit in compute, AMD will hear them out too. We'll see if that hardware customization strategy works outside of consoles.
[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by rcxdude on Tuesday June 25, 2024 @07:04PM (#40794702)
It certainly helps differentiate from NVIDIA's "Don't even think about putting our chips on a PCB we haven't vetted" approach.
[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by pjmlp on Wednesday June 26, 2024 @04:05AM (#40797493)
Yeah, but they will be using internal Microsoft and Meta software stacks, nothing that will dent CUDA.
[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by Rinzler89 on Tuesday June 25, 2024 @04:23PM (#40793159)
>I feel like people forget that AMD has huge contracts with Microsoft, Valve, Sony, etc to design consoles at scale.

Nobody forget that, just that those console chips are super low margins, which is why Intel and Nvidia stopped catering to that market after the Xbox/PS3 generations and only AMD took it up because they were broke and every penny mattered to them.

Nvidia did a brief stint with the Shield/Switch because they were trying to get into the Android/ARM space and also kinda gave up due to the margins.

[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by pjmlp on Wednesday June 26, 2024 @03:57AM (#40797435)
A market that keeps being discussed that is reaching its end, as newer generations aren't that much into traditional game consoles, and both Sony and Microsoft[0] have to reach out to PCs and mobile devices, to achieve sales growth.

Among the gamer community the discussion of this being the last generation keeps poping up.

[0] - Nintendo is more than happy to keep redoing their hit franchaises, in good enough hardware.

[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by 0cf8612b2e1e on Tuesday June 25, 2024 @12:29PM (#40790491)
On the other hand, AMD has had a decade of watching CUDA eat their lunch and done basically nothing to change the situation.
[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by bee_rider on Tuesday June 25, 2024 @01:02PM (#40790887)
AMD tries to compete in hardware with Intel’s CPUs and Nvidia’s GPUs. They have to slack somewhere, and software seems to be where. It isn’t any surprise that they can’t keep up on every front, but it does mean they can freely bring in partners whose core competency is software and work with them without any caveats.

Not sure why they haven’t managed to execute on that yet, but the partners must be pretty motivated now, right? I’m sure they don’t love doing business at Nvidia’s leisure.

[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by bobsondugnut on Tuesday June 25, 2024 @01:11PM (#40791007)
when was the last time AMD hardware was keeping up with NVIDIA? 2014?
[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by 0cf8612b2e1e on Tuesday June 25, 2024 @01:41PM (#40791289)
Been a while since AMD had the top tier offering, but it has been trading blows in the middle tier segment the entire time. If you are just looking for a gamer card (ie not max AI performance), the AMD is typically cheaper and less power hungry than the equivalent Nvidia.
[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by aurareturn on Tuesday June 25, 2024 @02:04PM (#40791568)
It’s trading blows because AMD sells their cards at lower margins in the midrange and Nvidia lets them.
[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by bee_rider on Wednesday June 26, 2024 @11:14AM (#40800980)
But, the fact that Nvidia cards command higher margins also reflects their better software stack, right? Nvidia “lets them” trade blows in the midrange, or, equivalently, Nvidia is receiving the reward of their software investments: even their midrange hardware commands a premium.
[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by bobsondugnut on Tuesday June 25, 2024 @02:03PM (#40791555)
> the AMD is typically cheaper and less power hungry than the equivalent Nvidia

cheaper is true, but less power hungry is absolutely not true, which is kind of my point.

[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by dralley on Tuesday June 25, 2024 @06:25PM (#40794374)
It was true with RDNA 2. RDNA 3 regressed on this a bit, supposedly there was a hardware hiccup that prevented them from hitting frequency and voltage targets that they were hoping to reach.

In any case they're only slightly behind, not crazy far behind like Intel is.

[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by bee_rider on Tuesday June 25, 2024 @02:08PM (#40791601)
The MI300X sounds like it is competitive, haha
[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by bobsondugnut on Tuesday June 25, 2024 @02:43PM (#40792000)
competitive with H100 for inference. a 2 year old product on just one half of the ML story. H200 (and potentially B100) is the appropriate comparison based on their production in volume.
[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by pjmlp on Wednesday June 26, 2024 @04:06AM (#40797498)
Hardware is useless without software to make it show off.
[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by adabyron on Tuesday June 25, 2024 @12:32PM (#40790528)
I have read in a few places that Microsoft is using AMD for inference to run ChatGPT. If I recall they said the price/performance was better.

I'm curious if that's just because they can't get enough Nvidia GPUs or if the price/performance is actually that much better.

[ Parent | on HN ]
Re:Testing AMD's Giant MI300X by atq2119 on Tuesday June 25, 2024 @01:39PM (#40791257)
Most likely it really is better overall.

Think of it this way: AMD is pretty good at hardware, so there's no reason to think that the raw difference in terms of flops is significant in either direction. It may go in AMD's favor sometimes and Nvidia's other times.

What AMD traditionally couldn't do was software, so those AMD GPUs are sold at a discount (compared to Nvidia), giving you better price/performance if you can use them.

Surely Microsoft is operating GPUs at large enough scale that they can pay a few people to paper over the software deficiencies so that they can use the AMD GPUs and still end up ahead in terms of overall price/performance.

[ Parent | on HN ]