News for hackers. Stuff that matters.
Re:Run LLMs on Apple Neural Engine (ANE) by seanmcdirmid on Saturday May 03, 2025 @03:23PM (#43881418)
Unless it’s a large model that doesn’t fit in the 5090, bust that’s no longer a $4k macstudio I think.
Re:Run LLMs on Apple Neural Engine (ANE) by voidspark on Saturday May 03, 2025 @03:58PM (#43881728)
$4k will get you a 96 GB Mac Studio with M3 Ultra (819 GB/sec).

That's 3x the RAM of the 5090.

Re:Run LLMs on Apple Neural Engine (ANE) by mdp2021 on Saturday May 03, 2025 @07:47PM (#43883295)
> That's 3x the RAM of the 5090

And a bit less than half the bandwidth (saying for completeness).

Re:Run LLMs on Apple Neural Engine (ANE) by behnamoh on Saturday May 03, 2025 @03:30PM (#43881469)
that's orthogonal to the speed discussion.

also, the GP was mostly testing models that fit in both 5090 and Mac Studio.