News for hackers. Stuff that matters.
Re:Zamba2-7B by hidelooktropic on Monday October 14, 2024 @11:06PM (#41844520)
To run gguf files? LM Studio for one. I think recurse on macos as well and probably some others.
Re:Zamba2-7B by x_may on Tuesday October 15, 2024 @07:10AM (#41847312)
As another commenter said, this has no GGUF because it’s partially mamba based which is unsupported in llama.cpp
Re:Zamba2-7B by xyc on Tuesday October 15, 2024 @05:08PM (#41853147)
dev of https://recurse.chat/ here, thanks for mentioning! rn we are focusing on features like shortcuts/floating window, but will look into support this in some time. to add to the llama.cpp support discussion, it's also worth noting that llama.cpp does not yet support gpu for mamba models https://github.com/ggerganov/llama.cpp/issues/6758