News for hackers. Stuff that matters.

A comment on Running a 180B parameter LLM on a single Apple M2 Ultra. Parent comment

Re:Running a 180B parameter LLM on a single Apple M2 Ultra by ronsor on Thursday September 07, 2023 @04:57PM (#37425474)
The problem with that is LLM speed is mostly bottlenecked by memory bandwidth. Slower RAM means worse performance.
[ Parent | on HN ]

There are no comments on this comment yet.