Re:Meta AI releases Code Llama 70B
by theLiminator on Tuesday January 30, 2024 @01:57PM
(#39194111)
Do you know how much vram is required?
[ Parent
| on HN ]
if you are asking about Apple silicon based Macs, they have integerated GPUs and do not have dedicated graphics memory (UMA)
For running 4 bit quantized model, with 70B parameters you will need around 35G Ram to load it in the memory. So I sould say a Mac with at least 48G memory. That is M3 Max.