News for hackers. Stuff that matters.
Re:Triton: Open-Source GPU Programming for Neural Networks by shubuZ on Wednesday July 28, 2021 @01:07PM (#27986176)
Which other hardware vendor provides the level of performance that Nvidia's GPU provide? Wasnt the benevolence on single (or couple) manufacturer(s) true in 90s, 2020s?
Re:Triton: Open-Source GPU Programming for Neural Networks by croes on Wednesday July 28, 2021 @01:59PM (#27987104)
It's not about performance but open standards. Remember Oracle vs. Google, at some time in the future NVidia could decide to get money out of CUDA.
Re:Triton: Open-Source GPU Programming for Neural Networks by einpoklum on Wednesday July 28, 2021 @04:15PM (#27989150)
Well, yes and no. Neither AMD nor NVIDIA are committed to open, libre, standards - they've both been "betraying" OpenCL (just like Apple and Intel, I should mention). So we have AMD's not-open (?) ROCm and NVIDIA's not-open CUDA.

Also, if we only cared about open standard we'd simply not use a GPU at all and settle RISC-V chips :-)

Re:Triton: Open-Source GPU Programming for Neural Networks by hhh on Wednesday July 28, 2021 @02:23PM (#27987529)
Tenstorrent.
Re:Triton: Open-Source GPU Programming for Neural Networks by dragontamer on Wednesday July 28, 2021 @02:32PM (#27987684)
AMD's MI100 is slightly faster than NVidia A100 for double-precision FLOPs at slightly lower costs. Good enough for Oak Ridge National Labs (Frontier Supercomputer), to say the least.

NVidia is faster at 4x4 16-bit matrix multiplications (common in Tensor / Deep Learning stuff), but MI100 still has 4x4 16-bit matrix multiplication instructions and acceleration. Its not far behind, and the greater 64-bit FLOPs is enough to win in scientific fields.

Re:Triton: Open-Source GPU Programming for Neural Networks by gnufx on Wednesday July 28, 2021 @03:00PM (#27988144)
Is Frontier known to be using M100? I haven't been following along, but it was trailed as using purpose-built GPUs (presumably also for Sierra).
Re:Triton: Open-Source GPU Programming for Neural Networks by dragontamer on Wednesday July 28, 2021 @03:05PM (#27988217)
Frontier is going to be an AMD-GPU for sure. Whether its MI100 or the next generation MI200 is still not really known and subject to rumors. I'm currently under the assumption that its MI100 however, but now that you press me on that detail I admit that I'm somewhat assuming. Frontier will be using ROCm in any case.

El Capitan (Lawrence Rivermore National Labs) is MI100 and specified as such. So I'll "retreat" my argument to that more easily verified position. Still a top supercomputer in the world, and more is known about El Capitan than Frontier.

Re:Triton: Open-Source GPU Programming for Neural Networks by gnufx on Wednesday July 28, 2021 @05:08PM (#27989737)
Yes, I agree with the basic point. Where is the spec for El Capitain, then? I don't remember ever finding it, and I rather assumed it would wait for possible lessons from the Frontier hardware.

It's possibly worth saying that these things presumably are competitive with nvlink-based multi-GPU, and appear to support unified memory, but I don't know how that compares with ac922s. Not that UM is relevant for what I know of usage on our system....

Re:Triton: Open-Source GPU Programming for Neural Networks by dragontamer on Wednesday July 28, 2021 @05:44PM (#27990132)
Wow. This is quite frustrating.

I have plenty of links, but they're all from non-primary source newsites, like servethehome or (worse) wcftech. Those are more "casual" blogs that aren't worth quoting IMO and are closer to rumormills (even if they sometimes have good information).

I can't find a single spec from AMD's website or ORNL, or the Department of Energy (primary sources: the ones we know would tell the truth on this matter).

The closest I got is this: https://www.amd.com/system/files/documents/oak-ridge-nationa...

Which is an MI100 test that ORNLs did in "preparation" for Frontier. It doesn't seem like Frontier nor El Capitan have had their specs officially released yet.

Re:Triton: Open-Source GPU Programming for Neural Networks by N1H1L on Wednesday July 28, 2021 @07:45PM (#27991148)
Re:Triton: Open-Source GPU Programming for Neural Networks by nl on Wednesday July 28, 2021 @09:14PM (#27991700)
Still doesn't have specs I can see!