News for hackers. Stuff that matters.
Re:Facebook LLAMA is being openly distributed via torrents by terafo on Friday March 03, 2023 @02:07PM (#35013690)
128 gigs is more than enough to load 13B model into. Pytorch has M1 support for some time now so CUDA isn't required.
Re:Facebook LLAMA is being openly distributed via torrents by eightysixfour on Friday March 03, 2023 @02:30PM (#35013947)
Does M1 use system memory as VRAM as well?
Re:Facebook LLAMA is being openly distributed via torrents by jml7c5 on Friday March 03, 2023 @03:16PM (#35014478)
Yes, it's all unified memory.

As much as I dislike the loss of socketed RAM, I have to say it's working out well for Apple users so far. I wonder if CXL will change the situation for consumer devices or if it will only be useful at the scale of a server rack.

Re:Facebook LLAMA is being openly distributed via torrents by MacsHeadroom on Saturday March 04, 2023 @09:25PM (#35026423)
13B fits on a single 3090 (24GB) in int8.

128 gigs might even be enough for 65B, if slowly.