News for hackers. Stuff that matters.
Re:Ollama's new engine for multimodal models by magicalhippo on Friday May 16, 2025 @05:03PM (#44009809)
They can of course meaningfully contribute new C++ code to llama.cpp, which they then could later use downstream in Go.

What they cannot meaningfully do is write Go code that solves their problems and upstream those changes to llama.cpp.

The former requires they are comfortable writing C++, something perhaps not all Go devs are.

Re:Ollama's new engine for multimodal models by refulgentis on Friday May 16, 2025 @08:12PM (#44010943)
I'd love to be able to take this into account, step back, and say "Ah yes - there is non-zero parobability they are materially incapable of doing it" - in practice, if you're comfortable writing SWA in Go, you're going to be comfortable writing it in C++, and they are writing C++ already.

(it's also worth looking at the code linked for the model-specific impls, this isn't exactly 1000s of lines of complicated code. To wit, while they're working with Georgi...why not offer to help land it in llama.cpp?)

Re:Ollama's new engine for multimodal models by magicalhippo on Friday May 16, 2025 @10:44PM (#44011647)
Perhaps for SWA.

For the multimodal stuff it's a lot clear cut. Ollama used the image processing libraries from Go, while in llama.cpp they ended up rolling their own image processing routines.

Re:Ollama's new engine for multimodal models by refulgentis on Friday May 16, 2025 @10:57PM (#44011701)
Citation?

My groundbreaking implementation passes it RGB bytes, passes em through the image projector, and put the tokens in the prompt.

And I cannot imagine sure why the inference engine would be more concerned with it than that.

Is my implementation a groundbreaking achievement worth rendering llama.cpp a footnote, because I use Dart image-processing libraries?

Re:Ollama's new engine for multimodal models by magicalhippo on Friday May 16, 2025 @11:24PM (#44011801)
> Citation?

https://github.com/ollama/ollama/issues/7300#issuecomment-24...

https://github.com/ggml-org/llama.cpp/blob/3e0be1cacef290c99...

Anyway my point was just that it's not as easy as just pushing a patch upstream, like it is in many other projects. It would require a new or different implementation.

Re:Ollama's new engine for multimodal models by refulgentis on Saturday May 17, 2025 @12:13AM (#44011995)
I don't know what's going on, you seem to have linked to an out-of-thin-air comment saying it's impossible because they can't figure out how to write C++ only Go. I guess because you quoted it verbatim as an explain of why they make any contributions? We called that a tautology, back in my day. I vaguely remember being flummoxed by this comment at the time because llama.cpp already had image support.