News for hackers. Stuff that matters.
Re:Run LLMs on Apple Neural Engine (ANE) by imtringued on Sunday May 04, 2025 @04:46AM (#43885445)
ONNX is horrible for anything that has variable input shapes and that is why nobody uses it for LLMs. It fundamentally is poorly designed for anything that doesn't take a fixed size image.
Re:Run LLMs on Apple Neural Engine (ANE) by zozbot234 on Sunday May 04, 2025 @06:51AM (#43885959)
ANE itself is also limited to fixed computation "shapes" so I'm not sure how much that would matter practically.