News for hackers. Stuff that matters.
Re:Gemma 3 QAT Models: Bringing AI to Consumer GPUs by parched99 on Sunday April 20, 2025 @04:25PM (#43746296)
Resolving that issue, would help reduce (not eliminate) the size of the context. The model will still only just barely fit in 16 GB, which is what the parent comment asked.

Best to have two or more low-end, 16GB GPUs for a total of 32GB VRAM to run most of the better local models.

There are no comments on this comment yet.