I don't think "large projects" is realistic with a model that fits in ~8GB (I'm assuming you run stuff other than the model). And, Gemma 4 12B QAT at 4-bits is surely the smartest model in its size, but it shines at vision tasks rather than agentic tasks (though it is a good tool user and can do stuff like research, it's obviously not aimed at code).

You can almost always find free models on OpenRouter. Google AI Studio also has free usage of Gemma 4. Both are rate and usage limited which agentic use will probably chew up pretty quick, but you can usually find some pretty powerful models for free. If you rotate through different providers, I think it avoids the cap. Currently several Nemotron models, North Mini Code, Laguna models, Gemma 4 31b and MoE, Qwen 3 Next, and gpt-oss 120b, are all available free on OpenRouter...and better than anything you can run locally in ~8GB.

WOW. That was alot of models. I will be sure to look into those. What caught my eye was the North Mini Code. How is that at reason and context? I guess I will just go see. Thank you for the insight. I am building and learning at the same time, and I really need a model that wont chew up my poor little M1, but at the same time will understand my direction, my context, and also my explanation of need and then be ...' oh, I know what you mean..' then help vibe with me to create it. Qwen was a code model but hallucinated and was never really good at contextual understanding. Gemma seems to get it but isnt necessarily a coding model.(As you said) And I just cant be bogged down with all these fees to get what I need on my journey. So onward on the model search I suppose. Thank you very much for the time.

North Mini Code is on par with the similar-sized MoE Qwen 3.6 and Gemma 4. It benches a little better than Gemma 4 26b a4b and a little worse than Qwen 3.6 35b a3b for most coding-related and agentic tasks. For my use case, or at least the use case I've tested, it is a little worse than both for surfacing security bugs, but all of the MoE models at this size are pretty bad at finding security bugs (they hallucinate a lot of false positives, which degrades the value of their real bugs dramatically).

Poolside Laguna XS.2 is another in this category (30B-35B MoE, reasonably competitive on coding related benchmarks). Also free on OpenRouter. But, also, it's bigger brother, Laguna M.1 (225B A23B MoE), is also free on OpenRouter last time I checked. Worth a look. https://openrouter.ai/poolside/laguna-m.1:free

The good thing about all the 30-ish MoE models is you can run them on any 32GB GPU, even old ones, at a very comfortable speed. A 24GB GPU can run the 4-bit quantizations if you use a quantized K/V cache. That's why there are so many of them. It's the sweet spot for "good enough to be useful for some coding tasks, small enough to fit on the GPU a lot of people have".

The cheapest not rate-limited options that are actually pretty competitive with the frontiers are from DeepSeek and MiMo. DeepSeek V4 Flash and Pro are extremely cheap, their caching is the best in the industry (and their cached tokens are even cheaper), and Reasonix is an excellent CLI harness that is designed around maximizing cacheability of DeepSeek models, specifically. I used it for an hour last night and spent something like three cents. MiMo has token plans that are a pretty good deal (though confusing...the token plan buys credits, and credits are not a whole token, so you get billions of credits on the token plan for a few bucks, but it chews through it at a rate faster than 1 credit per token). But, DeepSeek V4 Pro is a consistently better model than MiMo.

I appreciate the insight. I might look at these a bit later... because I am swamped with this new build Im putting together. I havent yet found my so-called 'sweet spot' yet for larger tasks. I checked out the gemma4 variant the one or two times just for contextual reasoning and smaller build capabilities, but Ive only been testing single use isolated tool tests with my mcp pointed at it lately to ensure tools I've installed actually work before I incorporate them into the mcp server. No plug or anything, but just so you can get a feel of what I'm doing, I initially set out to learn coding. Then I fell into trying to build my own local model to help teach me coding (and anything else related) because tutorials and all these fancy webinars and such just put me to sleep. Once I discovered all of this AI stuff...? My interest in cybersecurity just skyrocketed. So....no life story or anything) ha... I started to build this mcp, pointed at LM Studio with qwen loaded as my first one. As I began to familiarise more I realised...I need a model that is free, local, will understand context, reasoning, can code, de-bug,vuln discovery- etc. etc. All of this sent me on this whole cybersecurity rabbit hole - as it does with any tech stuff, and now along with my need for a model to teach me coding(Python) I wanted a model to also HELP me build ITSELF basically. Because so far, Ive used good old Claude- Sonnet-5 straight from the mobile app for said task. Dont laugh. And I show the build,and upgrades of the mcp to GEMMA (initially qwen) and get fedback against the Cluade assisted builds. So I apologise for ruining your day, but... thats where Im headed. I need a model that will vibe with me, build with me, reason, comprehend context and also... not chew up my m1 16gb, or token usage. Well, if its free I suppose tokens arent a thing but they do sort of matter still. Thanks again. Your information is much appreciated and invaluable.

Sorry, I also forgot to say, to the rest of your comment: Its actually 16gb M1. And yes the entire operation is, all running at the same time. Kali/UTM with an mcp server pointed at my LMstudio with gemma loaded as the local server. And maybe some spotify to lighten the mood while I work. But I learned not to have multiple things running, REAL QUICK. So ultimately I am running pretty healthy and no sort of crashes or choke points. I ran a 14b QWEN in LM Studio w/ no issue. Any higher and I think I would fall off a cliff somewhere.