News for hackers. Stuff that matters.
Re:Ollama's new engine for multimodal models by refulgentis on Friday May 16, 2025 @08:12PM (#44010943)
I'd love to be able to take this into account, step back, and say "Ah yes - there is non-zero parobability they are materially incapable of doing it" - in practice, if you're comfortable writing SWA in Go, you're going to be comfortable writing it in C++, and they are writing C++ already.

(it's also worth looking at the code linked for the model-specific impls, this isn't exactly 1000s of lines of complicated code. To wit, while they're working with Georgi...why not offer to help land it in llama.cpp?)

Re:Ollama's new engine for multimodal models by magicalhippo on Friday May 16, 2025 @10:44PM (#44011647)
Perhaps for SWA.

For the multimodal stuff it's a lot clear cut. Ollama used the image processing libraries from Go, while in llama.cpp they ended up rolling their own image processing routines.

Re:Ollama's new engine for multimodal models by refulgentis on Friday May 16, 2025 @10:57PM (#44011701)
Citation?

My groundbreaking implementation passes it RGB bytes, passes em through the image projector, and put the tokens in the prompt.

And I cannot imagine sure why the inference engine would be more concerned with it than that.

Is my implementation a groundbreaking achievement worth rendering llama.cpp a footnote, because I use Dart image-processing libraries?

Re:Ollama's new engine for multimodal models by magicalhippo on Friday May 16, 2025 @11:24PM (#44011801)
> Citation?

https://github.com/ollama/ollama/issues/7300#issuecomment-24...

https://github.com/ggml-org/llama.cpp/blob/3e0be1cacef290c99...

Anyway my point was just that it's not as easy as just pushing a patch upstream, like it is in many other projects. It would require a new or different implementation.

Re:Ollama's new engine for multimodal models by refulgentis on Saturday May 17, 2025 @12:13AM (#44011995)
I don't know what's going on, you seem to have linked to an out-of-thin-air comment saying it's impossible because they can't figure out how to write C++ only Go. I guess because you quoted it verbatim as an explain of why they make any contributions? We called that a tautology, back in my day. I vaguely remember being flummoxed by this comment at the time because llama.cpp already had image support.