Re:Hardware Acceleration of LLMs: A comprehensive survey and...
by fsndz on Saturday September 07, 2024 @06:39PM
(#41476889)
True. Samsung's Dr. Jung Bae Lee was also talking about that recently.
"rapid growth of AI models is being constrained by a growing disparity between compute performance and memory bandwidth. While next-generation models like GPT-5 are expected to reach an unprecedented scale of 3-5 trillion parameters, the technical bottleneck of memory bandwidth is becoming a critical obstacle to realizing their full potential."
"rapid growth of AI models is being constrained by a growing disparity between compute performance and memory bandwidth. While next-generation models like GPT-5 are expected to reach an unprecedented scale of 3-5 trillion parameters, the technical bottleneck of memory bandwidth is becoming a critical obstacle to realizing their full potential."
https://www.lycee.ai/blog/2024-09-04-samsung-memory-bottlene...