News for hackers. Stuff that matters.

A comment on ONNX Runtime and CoreML May Silently Convert Your Model to FP16. Parent comment

Re:ONNX Runtime and CoreML May Silently Convert Your Model t... by pzo on Monday December 22, 2025 @04:48AM (#46352681)
However one benefits of CoreML - it is the only way to be able for 3rd party to execute on ANE (Apple Neural Engine aka NPU). ANE for some models can execute even faster than GPU/MPS and consume even less battery.

But I agree CoreML in ONNX Runtime is not perfect - most of the time when I tested some models there were too many partitioning and whole graph was running slower compare when using only model in just CoreML format.

[ Parent | on HN ]
Re:ONNX Runtime and CoreML May Silently Convert Your Model t... by Two_hands on Monday December 22, 2025 @05:05AM (#46352785)
To be honest it's a shame the whole thing is closed up, I guess it's to be expected from Apple, but I reckon CoreML would be benefit a lot from at least exposing the internals/allowing users to define new ops.

Also, the ANE only allows some operators to be ran on it right? There's very little transparency/control on what can be offloaded to it and cannot which makes using it difficult.

[ Parent | on HN ]