ONNX Runtime and CoreML May Silently Convert Your Model to FP16

Re:ONNX Runtime and CoreML May Silently Convert Your Model t... by Two_hands on Monday December 22, 2025 @04:07AM (#46352433)

Thank you for reading.

Also generally I think CoreML isn't the best. The best solution for ORT would probably be to introduce a pure MPS provider (https://github.com/microsoft/onnxruntime/issues/21271), but given they've already bought into CoreML the effort may not be worth the reward for the core team. Which fair enough as it's a pretty mammoth task

[ Parent | on HN ]

Re:ONNX Runtime and CoreML May Silently Convert Your Model t... by pzo on Monday December 22, 2025 @04:48AM (#46352681)

However one benefits of CoreML - it is the only way to be able for 3rd party to execute on ANE (Apple Neural Engine aka NPU). ANE for some models can execute even faster than GPU/MPS and consume even less battery.

But I agree CoreML in ONNX Runtime is not perfect - most of the time when I tested some models there were too many partitioning and whole graph was running slower compare when using only model in just CoreML format.

[ Parent | on HN ]

Re:ONNX Runtime and CoreML May Silently Convert Your Model t... by Two_hands on Monday December 22, 2025 @05:05AM (#46352785)

To be honest it's a shame the whole thing is closed up, I guess it's to be expected from Apple, but I reckon CoreML would be benefit a lot from at least exposing the internals/allowing users to define new ops.

Also, the ANE only allows some operators to be ran on it right? There's very little transparency/control on what can be offloaded to it and cannot which makes using it difficult.

[ Parent | on HN ]