News for hackers. Stuff that matters.
Re:AMD MI300x GPUs with GEMM tuning improves throughput and ... by fancyfredbot on Saturday June 29, 2024 @07:34PM (#40834062)
The numbers in this article don't make sense. They aren't consistent with the hardware (they seem to show the weights being loaded faster than mi300x peak memory bandwidth) and they aren't self consistent (70B models running only 2x slower than 7B models).

I'm not sure what's going on here but something is not right.

Re:AMD MI300x GPUs with GEMM tuning improves throughput and ... by Lindon4290 on Saturday June 29, 2024 @08:00PM (#40834187)
Yeah, I've asked questions along those lines as well. Something sketchy is going on.

See here - https://news.ycombinator.com/item?id=40833109