Re:Devstral
by johnQdeveloper on Thursday May 22, 2025 @12:23AM
(#44058765)
https://ollama.com/library/devstral
I believe its just a HTTP wrapper and terminal wrapper around llama.cpp with some modifications/fork.
Re:Devstral
by throwaway314155 on Thursday May 22, 2025 @12:38AM
(#44058830)
Does ollama have support for cpu offloading?
Re:Devstral
by johnQdeveloper on Thursday May 22, 2025 @01:49AM
(#44059120)
> Does ollama have support for cpu offloading?
https://www.reddit.com/r/ollama/comments/1df757o/high_cost_o...
https://github.com/ollama/ollama/issues/8291
Yes.
A perfect blend of LMGTFY and helpfulness. :)
Re:Devstral
by johnQdeveloper on Thursday May 22, 2025 @04:29AM
(#44059984)
lol. I try not to be a total asshole, it sometime even works! :)
Good luck to you mate with your life :)
What're you using for this? llama.cpp? Have a 12GB card (rtx 4070) i'd like to try it on.