News for hackers. Stuff that matters.
Re:Stop Using Ollama by teekert on Thursday April 16, 2026 @03:55AM (#47790000)
Was hoping it was so easy :) But I probably need to look into it some more.

llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma4' llama_model_load_from_file_impl: failed to load model

Re:Stop Using Ollama by roosgit on Thursday April 16, 2026 @04:40AM (#47790325)
I just hit that error a few minutes ago. I build my llama.cpp from source because I use CUDA on Linux. So I made the mistake of trying to run Gemma4 on an older version I had and I got the same error. It’s possible brew installs an older version which doens’t support Gemma4 yet.
Re:Stop Using Ollama by teekert on Thursday April 16, 2026 @04:50AM (#47790396)
Ah it was indeed just that!

I'm now on:

$ llama --version version: 8770 (82764d8) built with GNU 15.2.0 for Linux x86_64

(From Nix unstable)

And this works as advertised, nice chat interface, but no openai API I guess, so no opencode...

Re:Stop Using Ollama by homarp on Thursday April 16, 2026 @05:00AM (#47790466)
check on same port, there is an OpenAI API https://github.com/ggml-org/llama.cpp/tree/master/tools/serv...
Re:Stop Using Ollama by teekert on Thursday April 16, 2026 @05:33AM (#47790740)
Good stuff, thanx!
Re:Stop Using Ollama by zozbot234 on Thursday April 16, 2026 @04:45AM (#47790362)
And that's exactly why llama.cpp is not usable by casual users. They follow the "move fast and break things" model. With ollama, you just have to make sure you're getting/building the latest version.
Re:Stop Using Ollama by Eisenstein on Thursday April 16, 2026 @06:20AM (#47791046)
Its not possible to run the latest model architectures without 'moving fast'. The only thing broken here is that they are trying to use an old version with a new model.
Re:Stop Using Ollama by cyanydeez on Thursday April 16, 2026 @06:38AM (#47791160)
and Ollama suffered the same fate when wanting to try new models
Re:Stop Using Ollama by Eisenstein on Thursday April 16, 2026 @09:31AM (#47792711)
What fate?
Re:Stop Using Ollama by cyanydeez on Thursday April 16, 2026 @01:18PM (#47796560)
the impedance mismatch between when models are released and the capability of Ollama and other servers capability for use.
Re:Stop Using Ollama by Eisenstein on Thursday April 16, 2026 @02:17PM (#47797361)
I'm a bit unsure what that has to do with someone running an outdated version of the program while trying to use a model that is supported in the latest release.
Re:Stop Using Ollama by adrian_b on Thursday April 16, 2026 @07:14AM (#47791422)
As it has been discussed in a few recent threads on HN, whenever a new model is released, running it successfully may need changes in the inference backends, such as llama.cpp.

There are 2 main reasons. One is the tokenizer, where new tokenizer definitions may be mishandled by the older tokenizer parsers.

The second reason is that each model may implement differently the tool invocations, e.g. by using different delimiter tokens and different text layouts for describing the parameters of a tool invocation.

Therefore running the Gemma-4 models encountered various problems during the first days after their release, especially for the dense 31B model.

Solving these problems required both a new version of llama.cpp (also for other inference backends) and updates in the model chat template and tokenizer configuration files.

So anyone who wants to use Gemma-4 should update to the latest version of llama.cpp and to the latest models from Huggingface, because the latest updates have been a couple of days ago.