Re:Facebook LLAMA is being openly distributed via torrents
by terafo on Friday March 03, 2023 @02:07PM
(#35013690)
128 gigs is more than enough to load 13B model into. Pytorch has M1 support for some time now so CUDA isn't required.
[ Parent
| on HN ]
Re:Facebook LLAMA is being openly distributed via torrents
by eightysixfour on Friday March 03, 2023 @02:30PM
(#35013947)
Does M1 use system memory as VRAM as well?
Re:Facebook LLAMA is being openly distributed via torrents
by jml7c5 on Friday March 03, 2023 @03:16PM
(#35014478)
Yes, it's all unified memory.
As much as I dislike the loss of socketed RAM, I have to say it's working out well for Apple users so far. I wonder if CXL will change the situation for consumer devices or if it will only be useful at the scale of a server rack.
Re:Facebook LLAMA is being openly distributed via torrents
by MacsHeadroom on Saturday March 04, 2023 @09:25PM
(#35026423)
13B fits on a single 3090 (24GB) in int8.
128 gigs might even be enough for 65B, if slowly.