News for hackers. Stuff that matters.
Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by brucethemoose2 on Friday February 23, 2024 @12:19PM (#39483415)
It's complicated, but basically because most are llama architecture. Meta all but set the standard for open source llms when they released llama1, and anyone trying to deviate from it has run into trouble because the models don't work with the hyper optimized llama runtumes.

Also, there's a lot of magic going on behind the scenes with configs stored in gguf/huggingface format models, and the libraries that use them. There are different tokenizers, but they mostly follow the same standards.

Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by null_point on Friday February 23, 2024 @03:43PM (#39485855)
Re:Gemma.cpp: lightweight, standalone C++ inference engine f... by null_point on Saturday February 24, 2024 @09:45AM (#39491805)
Hey, c'mon now. Just being playful about the "magic" string used in GGUF files to detect that it is in-fact a GGUF file.