Re:Running a 180B parameter LLM on a single Apple M2 Ultra
by nico on Thursday September 07, 2023 @12:23PM
(#37421363)
That’s for LLMs, but at the same time, there are other types of models coming out
Wouldn’t be surprised if we get small models that can run locally on a phone and just retrieve data from the network as needed (without sending your data out), within the next couple of years
Wouldn’t be surprised if we get small models that can run locally on a phone and just retrieve data from the network as needed (without sending your data out), within the next couple of years