Re:AMD MI300x GPUs with GEMM tuning improves throughput and ...
by latchkey on Saturday June 29, 2024 @02:45PM
(#40832605)
This is why I sponsored doing the chipsandcheese tests on my hardware. That instigated Elio to up the game even further.
All open source by the way.
Re:AMD MI300x GPUs with GEMM tuning improves throughput and ...
by fancyfredbot on Monday July 01, 2024 @04:26PM
(#40850275)
Thank you for sponsoring this. There's so little buzz about this hardware despite the fact it's clearly amazing for AI use cases. I don't understand why not. Maybe this is why Nvidia is the most valuable company in the world - nobody can be bothered to try a competitor.
300 tokens/s with bs=1 for a llama-2 70B on a single card is no joke.