News for hackers. Stuff that matters.

A comment on You can now train a 70B language model at home. Parent comment

Re:You can now train a 70B language model at home by int_19h on Friday March 08, 2024 @02:38PM (#39645334)
They do, but for inference at least, it's memory bandwidth that is the primary limiting factor for home LLMs right now, not raw compute.
[ Parent | on HN ]
Re:You can now train a 70B language model at home by sroussey on Saturday March 09, 2024 @01:20PM (#39653661)
Wonder if the apple silicon ultra series will start using HBM3(e) on desktop in the future.
[ Parent | on HN ]