Re:Running a 180B parameter LLM on a single Apple M2 Ultra
by ViktorBash on Thursday September 07, 2023 @02:07PM
(#37423114)
It's refreshing to see how fast open LLMs are advancing in terms of the models available. A year ago I thought that besides for the novelty of it, running LLMs locally would be nowhere close to stuff like OpenAI's closed models in terms of utility.
As more and more models become open and are able to be run locally, the precedent gets stronger (which is good for the end consumer in my opinion).
As more and more models become open and are able to be run locally, the precedent gets stronger (which is good for the end consumer in my opinion).