Re:Run LLMs on Apple Neural Engine (ANE)
by ericboehs on Monday May 05, 2025 @09:23AM
(#43894870)
Interesting. Does this mean larger models could be ran on less memory? It looks like it uses 15-20x less memory. Could a 671B DeepSeek R1 be ran in just ~40-50GB of memory? It sounds like it'd be 1/3 as fast though (<1tk/sec).
[ Parent
| on HN ]
There are no comments on this comment yet.