Re:Hardware Acceleration of LLMs: A comprehensive survey and...
by iml7 on Saturday September 07, 2024 @08:24AM
(#41473523)
expensive
Re:Hardware Acceleration of LLMs: A comprehensive survey and...
by deepnotderp on Saturday September 07, 2024 @01:08PM
(#41474958)
It’s expensive, not needed for most consumer workloads and ironically, is actually often worse for latency for many patterns, even though it’s much higher bandwidth
You seldom see it.