Man, my M1 Pro unfortunately only has 16GB of RAM; it has been such a limiter to using local models. I'll give kapitsa a try, I see I should be able to squeeze some smaller models on
kapitsa is customized to just a small selection of models that are in vogue right now
compared to llama.cpp that tries to cater to all but does so only averagely
check the catalogue and see if one of the quantizations fit
try kapitsa as the inference engine with pi as the harness
It works really well