Ask HN: Local AI Models for Coding and General Tasks?

Posted by giantg2 on Thursday November 05, 2026 @11:21AM
from the ask-hn dept.

Hi,

I'm looking at Gemma and Qwen as a local AI. I wanted to try it out for coding but also some other basic tasks. My system has an AMD based GPU with 12GB VRAM. What are some considerations or other options that I should look at?

( 5 comments | on HN )

Re:Ask HN: Local AI Models for Coding and General Tasks? by giantg2 on Thursday November 05, 2026 @05:36PM (#50385489)

I was doing some more research and think Gemma 4 12B would be a good model for my uses. I like that it has audio support, which I might have a use case for. It seems like it's close to Claude code running on older version of Sonnet, which would be good for me.

Anybody have experience running Gemma 4 12B locally? How do you like it? Any similar alternatives I should consider?

[ Parent | on HN ]

Re:Ask HN: Local AI Models for Coding and General Tasks? by sebion7125 on Thursday November 05, 2026 @06:42PM (#50386193)

what is the ammount od system RAM you have? If you've got enough Strata with Qwen3.8 Flash Next could be a great option. For me it works great on Pi. I've developed an extension, that made it useable for very long chats and it now helps developing on huge Repos like GrandOrgue. Sometimes it beat Sonnet 5.5 in this configuration, but the most efficient way is to combine it with Opus 5.5, when it gets stuck, or its plans are not optimal. After having the Max Subscription for 2 month I'm now able to keep the Pro subscription and still get good results.

[ Parent | on HN ]

Re:Ask HN: Local AI Models for Coding and General Tasks? by giantg2 on Friday November 06, 2026 @07:59PM (#50400573)

Interesting, but my system ram isn't that much. I think I've decided to go with Ollama Gemma4:12B. I just need to decide if I want to hook it into local Open Code or local Claude Code. I'm leaning towards Claude if it doesn't send any usage data back to Anthropic.

[ Parent | on HN ]

Re:Ask HN: Local AI Models for Coding and General Tasks? by linesofcode on Friday November 06, 2026 @11:04AM (#50394115)

Just a neat tip, if you’re on the latest OSX there’s a built-in model that’s accessible from the command line by running the `fm` command line. It’s too weak for planning and coding, but it can be used for trivial tasks.

[ Parent | on HN ]

Re:Ask HN: Local AI Models for Coding and General Tasks? by giantg2 on Monday November 09, 2026 @07:12PM (#50434322)

Reporting back that OpenCode with Ollama runs Gemma4:12b and Qwen3:14b well with 32k context. I might be able to bump it to 64k context, but not sure that will make a significant difference in my small projects, so I prefer the wiggle room.

Tried some 30b-ish models and they would load, but context was unusably small at 4k. I might have been able to bump it to 8k, but that would likely have been too tight.

7700xt 12gb and 32gb RAM.

[ Parent | on HN ]