Re:Gemma3 – The current strongest model that fits on a singl...
by anon373839 on Wednesday March 12, 2025 @06:53AM
(#43341858)
I’d recommend trying it on Google AI Studio (aistudio.google.com). I am getting exceptional results on a handful of novel problems that require deep domain knowledge and structured reasoning. I’m not able to replicate this performance with Ollama, so I suspect something is a bit off.
[ Parent
| on HN ]
Re:Gemma3 – The current strongest model that fits on a singl...
by pzo on Wednesday March 12, 2025 @07:54AM
(#43342256)
Maybe model is sensitive to quantization, by default ollama quantize it significantly.
Re:Gemma3 – The current strongest model that fits on a singl...
by tarruda on Wednesday March 12, 2025 @10:32AM
(#43343683)
I tried ollama fp16 and it had the same issues.
Re:Gemma3 – The current strongest model that fits on a singl...
by tarruda on Wednesday March 12, 2025 @10:32AM
(#43343674)
Same experience here: On AI Studio, this is easily one of the strongest models I have used, including when compared to proprietary LLMs.
But ollama and openwebui performance is very bad, even when running the FP16 version. I also tried to mirror some of AI studio settings (temp 1 and top p 0.95) but couldn't get it to produce anything useful.
I suspect there's some bug in the ollama releases (possibly wrong conversation delimiters?). If this is fixed, I will definitely start using Gemma 3 27b as my main model.
Re:Gemma3 – The current strongest model that fits on a singl...
by anon373839 on Thursday March 13, 2025 @09:39AM
(#43353239)
Update: Unsloth is recommending a temperature of 0.1, not 1.0, if using Ollama. I don’t know why Ollama would require a 10x lower value, but it definitely helped. I also read some speculation that there might be an issue with the tokenizer.