Running a 180B parameter LLM on a single Apple M2 Ultra

Re:Running a 180B parameter LLM on a single Apple M2 Ultra by 6510 on Friday September 08, 2023 @02:28AM (#37430155)

Okay ill tell you. You need to start a startup that sets up a good number of cameras at manual labor jobs. Most of the footage will be completely useless but every day you hit a once in a day event, every week you get a once in a week event, every month, every year, every decade etc! Then the guy working there for 40 years wacks pipe 224 with a hammer 50 cm from the outlet and production resumes.

The footage can be aggressively pruned to fit on the disk.

When the robot is delivered in 2033 it can easily figure out, from the footage, all these weird and rare edge cases.

The difference will be like that between a competent but new employee and someone with 10 years of experience.

I can see the Tesla bots disassembling the production line already. Or do you think it wont happen?

[ Parent | on HN ]

Re:Running a 180B parameter LLM on a single Apple M2 Ultra by Aerbil313 on Sunday September 10, 2023 @07:14AM (#37454779)

Transformers can and do forget.

[ Parent | on HN ]

Re:Running a 180B parameter LLM on a single Apple M2 Ultra by checkyoursudo on Wednesday September 13, 2023 @08:05AM (#37495611)

Assuming that this would work, which I am fine with granting for purposes of discussion, how does this method ever let you build anything new? Or make use of advances in production methods? Or completely reconfigure a production line because of some regulatory requirement?

[ Parent | on HN ]