News for hackers. Stuff that matters.
Re:Hardware Acceleration of LLMs: A comprehensive survey and... by DrNosferatu on Saturday September 07, 2024 @06:17AM (#41472919)
Why haven’t all GPUs migrated to HBMx?

You seldom see it.

Re:Hardware Acceleration of LLMs: A comprehensive survey and... by iml7 on Saturday September 07, 2024 @08:24AM (#41473523)
expensive
Re:Hardware Acceleration of LLMs: A comprehensive survey and... by deepnotderp on Saturday September 07, 2024 @01:08PM (#41474958)
It’s expensive, not needed for most consumer workloads and ironically, is actually often worse for latency for many patterns, even though it’s much higher bandwidth
Re:Hardware Acceleration of LLMs: A comprehensive survey and... by ska on Saturday September 07, 2024 @02:06PM (#41475353)
why is that ironic? Many ways to increase bandwidth come at the cost of latency...