The AMD Radeon Instinct MI300A's Giant Memory Subsystem

Re:The AMD Radeon Instinct MI300A's Giant Memory Subsystem by UncleOxidant on Saturday January 18, 2025 @01:00PM (#42750062)

Thanks for that link. I'm interested in either getting the HP Mini Z1 G1a or an NVidia Digits for LLM experimentation. The obvious advantage for the Digits is the CUDA ecosystem is much more tried & true for that kind of thing. But the disadvantage is trying to use it as a replacement for my current PC as well as the fact that it's going to run an already old version of Ubuntu (22.04) and you're dependent on Nvidia for updates.

[ Parent | on HN ]

Re:The AMD Radeon Instinct MI300A's Giant Memory Subsystem by lhl on Saturday January 18, 2025 @01:15PM (#42750176)

Yeah, I think anyone w/ old Jetsons knows what it's like to be left high and dry by Nvidia's embedded software support. Older models are basically just ewaste. Since the Digits won't be out until May, I guess there's enough time to wait and see - at least to get a sense of what the actual specs are. I have a feeling the FP16 TFLOPS and the MBW are going to be much lower than what people have been hyping themselves up for.

Sadly, my feeling is that the big Strix Halo SKUs (which have no scheduled release dates) aren't going to be competitively priced (they're likely to be at a big FLOPS/real-world performance disadvantage, and there's still the PITA factor), but there is something appealing about about the do-it-all aspect of it.

[ Parent | on HN ]

Re:The AMD Radeon Instinct MI300A's Giant Memory Subsystem by rbanffy on Saturday January 18, 2025 @03:29PM (#42751178)

DIGITS looks like a serious attempt, but they don’t have too much of an incentive to have people developing for older hardware. I wouldn’t expect them to supor it for more than five years. At least the underlying Ubuntu will last more than that and provide a viable work environment far beyond the time it gets really boring.

[ Parent | on HN ]

Re:The AMD Radeon Instinct MI300A's Giant Memory Subsystem by UncleOxidant on Saturday January 18, 2025 @05:07PM (#42751801)

If only they could get their changes upstreamed to Ubuntu (and possible kernel mods upstreamed), then we wouldn't have to worry about it.

[ Parent | on HN ]

Re:The AMD Radeon Instinct MI300A's Giant Memory Subsystem by rbanffy on Saturday January 18, 2025 @05:19PM (#42751874)

Getting their kernel mods upstreamed is very unlikely, but they might provide just enough you can build a new kernel with the same major version number.

[ Parent | on HN ]

Re:The AMD Radeon Instinct MI300A's Giant Memory Subsystem by KeplerBoy on Saturday January 18, 2025 @03:02PM (#42750988)

Who said anything about Ubuntu 22.04? I mean sure that's the newest release current jetpack comes with, but I'd be surprised if they shipped digits with that.

[ Parent | on HN ]

Re:The AMD Radeon Instinct MI300A's Giant Memory Subsystem by rbanffy on Saturday January 18, 2025 @03:26PM (#42751155)

Doesn’t DGX OS use the latest LTS version? Current should be 24.04.

[ Parent | on HN ]

Re:The AMD Radeon Instinct MI300A's Giant Memory Subsystem by KeplerBoy on Saturday January 18, 2025 @04:01PM (#42751395)

I wouldn't know. I only work with workstation or jetson stuff.

The DGX documentation and downloads aren't public afaik.

Edit: Nevermind, some information about DGX is public and they really are on 22.04, but oh well, the deep learning stack is guaranteed to run.

https://docs.nvidia.com/base-os/too

[ Parent | on HN ]