[dead]
Re:Run LLMs on Apple Neural Engine (ANE)
by imtringued on Sunday May 04, 2025 @04:46AM
(#43885445)
ONNX is horrible for anything that has variable input shapes and that is why nobody uses it for LLMs. It fundamentally is poorly designed for anything that doesn't take a fixed size image.