News for hackers. Stuff that matters.
Re:AMD MI300x GPUs with GEMM tuning improves throughput and ... by fancyfredbot on Saturday June 29, 2024 @07:02PM (#40833891)
For mixtral 7B latency, I think the explanation is just that chips and cheese used 128 token requests, and in the article it is 256 token requests

I still can't explain throughput differences though

There are no comments on this comment yet.