Re:Vision Now Available in Llama.cpp
by nolist_policy on Saturday May 10, 2025 @04:39AM
(#43944130)
On the other hand ollama supports iSWA for Gemma 3 while llama.cpp doesn't. iSWA reduces kv cache size to 1/6.
[ Parent
| on HN ]
Re:Vision Now Available in Llama.cpp
by vlovich123 on Saturday May 10, 2025 @05:18AM
(#43944300)
What’s iSWA? Can’t find any reference online
Re:Vision Now Available in Llama.cpp
by nolist_policy on Saturday May 10, 2025 @06:13AM
(#43944500)
interleaved sliding window attention
Re:Vision Now Available in Llama.cpp
by imtringued on Saturday May 10, 2025 @06:19AM
(#43944530)
Gemma 3 has some layers with a context size of 1024 tokens and others having full length. You need to read the Gemma technical report.