News for hackers. Stuff that matters.

macwebcomputing

Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel on Saturday October 14, 2023 @05:01PM (#37884244)
Interesting feedback but focus on cost is a old debate. Your feedback seems more about 3D than AI. A lot of developers also want a better user experience, reliability. My reply is that M1 and M2 are just the beginning as Apple is investing billion of dollars in R&D. Also, no PC laptops can do better than AS architecture today, at a lower cost than high-end PC laptops and more battery life. Pro servers is next step. M2 ultra is just a preview. Just saw two weeks ago, an engineer doing a demo of Llama 2 on a recent AMD laptop. he was complaining how slow it was compared to a Mac. Again, Apple is leading for laptops now, server is next. M3 will remove memory limitation.
Re:M2 Ultra can run 128 streams of Llama 2 7B in parallel on Thursday October 12, 2023 @11:47AM (#37858828)
macweb.com is also a solution :-)