News for hackers. Stuff that matters.
Re:Deploying Transformers on the Apple Neural Engine by hedgehog on Wednesday June 08, 2022 @04:30PM (#31673576)
Depending what you want to do you do you might find it faster to prototype with the Python bindings: https://coremltools.readme.io/docs/model-prediction
Re:Deploying Transformers on the Apple Neural Engine by pavlov on Wednesday June 08, 2022 @07:37PM (#31675871)
I don't want to use CoreML or Python, I wanted to learn about the lower-level implementation using Metal Performance Shaders. And the only available sample code doesn't even run.

This is sadly not a unique situation for Apple's more specialized public frameworks — they're more like semi-private because nobody outside Apple can make them work.

Re:Deploying Transformers on the Apple Neural Engine by hedgehog on Thursday June 09, 2022 @03:14AM (#31678391)
Ah, yeah, and that's an improvement over the historical state of their ML support. For inference the M1 is a nice platform (fastish and power efficient) but for research and training not fast or well supported enough to justify not using NVIDIA.