News for hackers. Stuff that matters.

A comment on You can now train a 70B language model at home. Parent comment

Re:You can now train a 70B language model at home by erichocean on Friday March 08, 2024 @03:48AM (#39639196)
Memory limited - an m2 ultra has >150GiB, a 4090 24GiB
[ Parent | on HN ]
Re:You can now train a 70B language model at home by lawlessone on Friday March 08, 2024 @10:21AM (#39641832)
So why is nobody doing this?
[ Parent | on HN ]
Re:You can now train a 70B language model at home by ttul on Friday March 08, 2024 @10:34AM (#39641983)
My personal experience with Apple Silicon and machine learning in comparison with Nvidia is that the libraries are often missing various features on Apple, leading to headaches when trying to use various libraries and tools. Apple is working to bridge the gap and I am excited for the gap to be closed because the memory bandwidth on big M2 and M3 machines is monstrous.
[ Parent | on HN ]
Re:You can now train a 70B language model at home by lawlessone on Friday March 08, 2024 @11:22AM (#39642585)
sounds similar to how people have described game dev for Mac. The hardware is there. It just isn't supported.
[ Parent | on HN ]
Re:You can now train a 70B language model at home by imhoguy on Friday March 08, 2024 @11:51AM (#39642948)
Apple could single-handedly kill consumer dGPU market if they have released proper low-level APIs for their M1/2/3. I feel they have some huge coming out in the pipe to tumble the "AI" market.
[ Parent | on HN ]