Re:PyTorch on Apple M1 MAX GPUs with SHARK – faster than Ten...
by noxa on Tuesday February 22, 2022 @06:34PM
(#30435398)
Correct! We're currently compiling to SPIR-V and then going through MoltenVK/spirv-cross to get to Metal - and our SPIR-V needs quite a bit of work (currently tuned for Mali, which is a very different architecture). The biggest limiter though is as you note Apple doesn't have a way to use the fancy hardware instructions whereas our CUDA version is using the Tensor Cores.
The good news (for us) is that this is all effectively -O1 today; there's still potential for 2-8x speedups over these numbers and that's not even factoring in the inaccessible HW features that maybe one day Apple will expose :crossed-fingers: :)
// IREE dev
Re:PyTorch on Apple M1 MAX GPUs with SHARK – faster than Ten...
by AceJohnny2 on Wednesday February 23, 2022 @01:35AM
(#30438359)
Any chance of using the Neural Engine (otherwise exposed by CoreML) ?
Re:PyTorch on Apple M1 MAX GPUs with SHARK – faster than Ten...
by magic_at_nodai on Wednesday February 23, 2022 @06:20PM
(#30448056)
unlikely since the interface from ANE is not public and it may change between hardware versions.
Honestly though I'm more concerned about the other aspects of apple's software: "The OSX Window Server crashes when all GPUs are used to the maximum"