Re:Ollama's new engine for multimodal models
by refulgentis on Friday May 16, 2025 @10:57PM
(#44011701)
Citation?
My groundbreaking implementation passes it RGB bytes, passes em through the image projector, and put the tokens in the prompt.
And I cannot imagine sure why the inference engine would be more concerned with it than that.
Is my implementation a groundbreaking achievement worth rendering llama.cpp a footnote, because I use Dart image-processing libraries?
Re:Ollama's new engine for multimodal models
by magicalhippo on Friday May 16, 2025 @11:24PM
(#44011801)
> Citation?
https://github.com/ollama/ollama/issues/7300#issuecomment-24...
https://github.com/ggml-org/llama.cpp/blob/3e0be1cacef290c99...
Anyway my point was just that it's not as easy as just pushing a patch upstream, like it is in many other projects. It would require a new or different implementation.
Re:Ollama's new engine for multimodal models
by refulgentis on Saturday May 17, 2025 @12:13AM
(#44011995)
I don't know what's going on, you seem to have linked to an out-of-thin-air comment saying it's impossible because they can't figure out how to write C++ only Go. I guess because you quoted it verbatim as an explain of why they make any contributions? We called that a tautology, back in my day. I vaguely remember being flummoxed by this comment at the time because llama.cpp already had image support.
For the multimodal stuff it's a lot clear cut. Ollama used the image processing libraries from Go, while in llama.cpp they ended up rolling their own image processing routines.