Re:Meta AI releases Code Llama 70B
by israrkhan on Wednesday January 31, 2024 @05:25PM
(#39210289)
if you are asking about Apple silicon based Macs, they have integerated GPUs and do not have dedicated graphics memory (UMA)
For running 4 bit quantized model, with 70B parameters you will need around 35G Ram to load it in the memory. So I sould say a Mac with at least 48G memory. That is M3 Max.
For running 4 bit quantized model, with 70B parameters you will need around 35G Ram to load it in the memory. So I sould say a Mac with at least 48G memory. That is M3 Max.