Re:Run LLMs on Apple Neural Engine (ANE)
by anemll on Sunday May 04, 2025 @10:28AM
(#43886921)
Right.I was thinking about it, you still need batch refill, however, Apple Core ML tools were failing for attention activations quantization. Long context, pre-fill is still compute bound.
[ Parent
| on HN ]
There are no comments on this comment yet.