Re:GitHub Copilot is now available for free
by indigodaddy on Thursday December 19, 2024 @08:54AM
(#42461387)
Thanks, I need to figure out something to run on M4 Mac Mini standard 16G
[ Parent
| on HN ]
Re:GitHub Copilot is now available for free
by int_19h on Thursday December 19, 2024 @12:09PM
(#42463483)
You should be looking at 7-8b sized models, then. https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct is considered pretty strong for its size. That said, you shouldn't expect more than glorified autocompletion at that point.
Re:GitHub Copilot is now available for free
by thot_experiment on Thursday December 19, 2024 @08:39PM
(#42467502)
I'm not familiar with the practical memory requirements on Macs but I suspect that with 16gb of integrated ram you won't have issues running 14B models even at q6_k, certainly would be fine at q4 and it's definitely going to be capable of writing code based on instruction as well as minor refactoring, generating docstrings etc.
Re:GitHub Copilot is now available for free
by int_19h on Thursday December 19, 2024 @09:55PM
(#42467856)
The model itself will fit just fine, of course, but you'll also want a large context for coding. And then since it's integrated RAM, it's also used by everything else running on your system - like, say, your IDE, your compiler etc, which are all fairly memory hungry.
Also keep in mind that, even though it's "unified memory", the OS enforces a certain quota for the GPU. If I remember correctly, it's something like 2/3 of the overall RAM.