News for hackers. Stuff that matters.
Re:Vision Now Available in Llama.cpp by nolist_policy on Saturday May 10, 2025 @04:39AM (#43944130)
On the other hand ollama supports iSWA for Gemma 3 while llama.cpp doesn't. iSWA reduces kv cache size to 1/6.
Re:Vision Now Available in Llama.cpp by vlovich123 on Saturday May 10, 2025 @05:18AM (#43944300)
What’s iSWA? Can’t find any reference online
Re:Vision Now Available in Llama.cpp by nolist_policy on Saturday May 10, 2025 @06:13AM (#43944500)
interleaved sliding window attention
Re:Vision Now Available in Llama.cpp by imtringued on Saturday May 10, 2025 @06:19AM (#43944530)
Gemma 3 has some layers with a context size of 1024 tokens and others having full length. You need to read the Gemma technical report.