Re:AMD MI300x GPUs with GEMM tuning improves throughput and ...
by fancyfredbot on Saturday June 29, 2024 @02:09PM
(#40832425)
I can't square the numbers in the two articles at all. Other sources seem closer to their optimised results than their unoptimised results:
[ Parent
| on HN ]
Re:AMD MI300x GPUs with GEMM tuning improves throughput and ...
by fancyfredbot on Saturday June 29, 2024 @07:02PM
(#40833891)
For mixtral 7B latency, I think the explanation is just that chips and cheese used 128 token requests, and in the article it is 256 token requests
I still can't explain throughput differences though