News for hackers. Stuff that matters.

A comment on Running a 180B parameter LLM on a single Apple M2 Ultra. Parent comment

Re:Running a 180B parameter LLM on a single Apple M2 Ultra by astrange on Thursday September 07, 2023 @07:28PM (#37427110)
I'd be impressed if you fit that into a phone.
[ Parent | on HN ]
Re:Running a 180B parameter LLM on a single Apple M2 Ultra by AnthonyMouse on Friday September 08, 2023 @03:44AM (#37430712)
It'll make phone calls. Just put a VoIP app on it.

Obviously what you can do in practice is put the interface on your phone. It doesn't have to run on battery to run locally.

[ Parent | on HN ]