News for hackers. Stuff that matters.

A comment on Apple M2 Ultra Isn't Faster Than Last Gen AMD&Intel Desktop CPUs, 50% Slower GPU. Parent comment

Re:Apple M2 Ultra Isn't Faster Than Last Gen AMD&Intel Deskt... by incrudible on Saturday June 10, 2023 @07:43PM (#36276502)
It is also the performance of 4 Mac Pros. GP was talking about tasks that dedicated GPUs supposedly could not handle. Who is is running models needing 60+ GB of VRAM on their laptops?
[ Parent | on HN ]
Re:Apple M2 Ultra Isn't Faster Than Last Gen AMD&Intel Deskt... by ElFitz on Saturday June 10, 2023 @09:36PM (#36277096)
Some have had fun running some of the larger Llama, Vicuna & Guanaco models on their MacBooks. Mostly to experiment with local inference.

Although quantisation has now lowered the required memory, I wouldn’t be surprised if it comes in handy again in the near future.

But today that’s quite a niche use case.

[ Parent | on HN ]