News for hackers. Stuff that matters.
Re:GitHub Copilot is now available for free by thot_experiment on Thursday December 19, 2024 @08:39PM (#42467502)
I'm not familiar with the practical memory requirements on Macs but I suspect that with 16gb of integrated ram you won't have issues running 14B models even at q6_k, certainly would be fine at q4 and it's definitely going to be capable of writing code based on instruction as well as minor refactoring, generating docstrings etc.
Re:GitHub Copilot is now available for free by int_19h on Thursday December 19, 2024 @09:55PM (#42467856)
The model itself will fit just fine, of course, but you'll also want a large context for coding. And then since it's integrated RAM, it's also used by everything else running on your system - like, say, your IDE, your compiler etc, which are all fairly memory hungry.

Also keep in mind that, even though it's "unified memory", the OS enforces a certain quota for the GPU. If I remember correctly, it's something like 2/3 of the overall RAM.