Re:My local model setup on an M4 Pro Mac Mini
by c16 on Wednesday September 02, 2026 @04:02AM
(#49533236)
True. This then boils down to a quality vs speed decision. the 3.8 27b is far better than 3.6 A3B from my experience. I'm happy taking the speed hit, given local models aren't as intelligent as frontier models. Anything that can get me closer to my CC experience both in reasonable speed and intelligence is worth it. With that said CC can also be slow at times, so it's locally the difference in experience is not always noticeable.
In this case the 35b a3b makes sense as it has a PP speed of around 800tok/s