News for hackers. Stuff that matters.
Re:Falcon 40B LLM (which beats Llama) now Apache 2.0 by takeiteasyy on Wednesday May 31, 2023 @08:37PM (#36146344)
This will require a cpp port first to run on e.g. apple silicon?
Re:Falcon 40B LLM (which beats Llama) now Apache 2.0 by rovr138 on Wednesday May 31, 2023 @09:23PM (#36146617)
This is using huggingface. It's python. There's no reason it wouldn't run on arm.

https://huggingface.co/tiiuae/falcon-40b

I'll also add that the fact something is in C++ doesn't mean it will run on arm or that it can be compiled in it.

Re:Falcon 40B LLM (which beats Llama) now Apache 2.0 by wsgeorge on Wednesday May 31, 2023 @09:57PM (#36146779)
No, but to benefit from the ggml ecosystem, yes. Someone's taking a stab at it: https://github.com/nikisalli/falcon.cpp