News for hackers. Stuff that matters.

A comment on Run LLMs on Apple Neural Engine (ANE). Parent comment

Re:Run LLMs on Apple Neural Engine (ANE) by anemll on Wednesday May 07, 2025 @12:14PM (#43917532)
M4 max should work at 120GB for ANE and 500+ for GPU. So GPU will be 3-4 times faster for anything over 1-3B. ANE is likely as fast for prefill due to higher FLOPs
[ Parent | on HN ]

There are no comments on this comment yet.