News for hackers. Stuff that matters.

A comment on Run LLMs on Apple Neural Engine (ANE). Parent comment

Re:Run LLMs on Apple Neural Engine (ANE) by thadk on Sunday May 04, 2025 @09:26AM (#43886591)
Sounds like M2-era onward have bfloat16: https://eclecticlight.co/2024/01/13/how-m1-macs-may-lag-behi...
[ Parent | on HN ]
Re:Run LLMs on Apple Neural Engine (ANE) by anemll on Sunday May 04, 2025 @10:22AM (#43886877)
Yes for GPU, however ANE only supports FP16 plus integers. M4/A17 added accelerated int8 that is twice faster than FP16
[ Parent | on HN ]