News for hackers. Stuff that matters.

A comment on Matrix multiplication inches closer to mythic goal. Parent comment

Re:Matrix multiplication inches closer to mythic goal by AnotherGoodName on Tuesday March 23, 2021 @01:57PM (#26558027)
Note that GPUS are almost always multiplying 4x4 matrices. Sure they multiply many matrices together but each is 4x4. The 4^3 complexity isn't at all an issue (64 steps) and the constant time for some of the n^2.x methods is high.
[ Parent | on HN ]
Re:Matrix multiplication inches closer to mythic goal by nicoburns on Tuesday March 23, 2021 @09:07PM (#26562350)
Presumably this only applies to GPUs being used for graphics, not GPUs being used for ML?
[ Parent | on HN ]
Re:Matrix multiplication inches closer to mythic goal by shaklee3 on Wednesday March 24, 2021 @01:08AM (#26563820)
This is not true. What is your source?
[ Parent | on HN ]
Re:Matrix multiplication inches closer to mythic goal by antixk on Wednesday March 24, 2021 @03:52AM (#26564568)
I guess OP meant that each tensor core in the latest NVIDIA GPUs fundamentaly performs 4x4 matrix multiplication [0].

[0] https://nla-group.org/2020/07/21/numerical-behaviour-of-tens...

[ Parent | on HN ]
Re:Matrix multiplication inches closer to mythic goal by shaklee3 on Wednesday March 24, 2021 @09:17AM (#26566786)
That's also not true. They do 4x4, 8x8, and 16x16.
[ Parent | on HN ]
Re:Matrix multiplication inches closer to mythic goal by AnotherGoodName on Wednesday March 24, 2021 @01:21PM (#26570203)
Which is fine for the point being made here. There's no issue with n^3 for small matrices.
[ Parent | on HN ]