News for hackers. Stuff that matters.
Re:Devstral by throwaway314155 on Thursday May 22, 2025 @12:20AM (#44058748)
> For people without a 24GB RAM video card, I've got an 8GB RAM one running

What're you using for this? llama.cpp? Have a 12GB card (rtx 4070) i'd like to try it on.

Re:Devstral by johnQdeveloper on Thursday May 22, 2025 @12:23AM (#44058765)
https://ollama.com/library/devstral

https://ollama.com/

I believe its just a HTTP wrapper and terminal wrapper around llama.cpp with some modifications/fork.

Re:Devstral by throwaway314155 on Thursday May 22, 2025 @12:38AM (#44058830)
Does ollama have support for cpu offloading?
Re:Devstral by johnQdeveloper on Thursday May 22, 2025 @01:49AM (#44059120)
Re:Devstral by taneq on Thursday May 22, 2025 @03:43AM (#44059677)
A perfect blend of LMGTFY and helpfulness. :)
Re:Devstral by johnQdeveloper on Thursday May 22, 2025 @04:29AM (#44059984)
lol. I try not to be a total asshole, it sometime even works! :)

Good luck to you mate with your life :)