News for hackers. Stuff that matters.
Re:PyTorch on Apple M1 MAX GPUs with SHARK – faster than Ten... by noxa on Tuesday February 22, 2022 @06:34PM (#30435398)
Correct! We're currently compiling to SPIR-V and then going through MoltenVK/spirv-cross to get to Metal - and our SPIR-V needs quite a bit of work (currently tuned for Mali, which is a very different architecture). The biggest limiter though is as you note Apple doesn't have a way to use the fancy hardware instructions whereas our CUDA version is using the Tensor Cores.

The good news (for us) is that this is all effectively -O1 today; there's still potential for 2-8x speedups over these numbers and that's not even factoring in the inaccessible HW features that maybe one day Apple will expose :crossed-fingers: :)

// IREE dev

Re:PyTorch on Apple M1 MAX GPUs with SHARK – faster than Ten... by AceJohnny2 on Wednesday February 23, 2022 @01:35AM (#30438359)
Any chance of using the Neural Engine (otherwise exposed by CoreML) ?
Re:PyTorch on Apple M1 MAX GPUs with SHARK – faster than Ten... by magic_at_nodai on Wednesday February 23, 2022 @06:20PM (#30448056)
unlikely since the interface from ANE is not public and it may change between hardware versions.