News for hackers. Stuff that matters.

A comment on AMD Inference. Parent comment

Re:AMD Inference by skirmish on Wednesday October 02, 2024 @05:52PM (#41725308)
My understanding is that ROCm contains all included kernels for each supported architecture, so it would have (made up):

  -- matrix multiply 2048x2048 for Navi 31,
  -- same for Navi 32,
  -- same for Navi 33,
  -- same for Navi 21,
  -- same for Navi 22,
  -- same for Navi 23,
  -- same for Navi 24, etc.
  -- matrix multiply 4096x4096 for Navi 31,
  -- ...
[ Parent | on HN ]
Re:AMD Inference by slavik81 on Wednesday October 02, 2024 @08:14PM (#41726136)
Your understanding is correct. Although, you wouldn't find Navi 22, 23 or 24 in the list because those particular architectures are not supported. Instead, you'd see Vega 10, Vega 20, Arcturus, Aldebaran, Aqua Vanjaram and sometimes Polaris.

We're working on a few different strategies to reduce the binary size. It will get worse before it gets better, but I think you can expect significant improvements in the future. There are lots of ways to slim the libraries down.

[ Parent | on HN ]