Re:Running a 180B parameter LLM on a single Apple M2 Ultra
by foobiekr on Friday September 08, 2023 @12:44AM
(#37429430)
That's literally my point. The problem is that there's a massive amount of hidden infrastructure behind those that you don't see and that "oh look everyone has a big model" isn't as impressive as it sounds.
[ Parent
| on HN ]
Re:Running a 180B parameter LLM on a single Apple M2 Ultra
by AnthonyMouse on Friday September 08, 2023 @03:52AM
(#37430771)
But the open source infrastructure is getting built too. And the infrastructure is mostly independent of the model. This is Falcon 180B running using the code from llama.cpp.