News for hackers. Stuff that matters.
Re:PyTorch on Apple M1 MAX GPUs with SHARK – faster than Ten... by olliej on Tuesday February 22, 2022 @06:28PM (#30435339)
My reading implied that they weren't able to use the apple hardware acceleration for AI, which given the massive perf delta I suspect the A100 is using. If they weren't using the A100's ML hardware I'd be both amazed by Nvidia's engineers, and also unhappy as it would be unfair for the test to not use the features of the hardware (this is a common refrain in some benchmarking circles where you disable features to get "fairness" in comparisons).

Honestly though I'm more concerned about the other aspects of apple's software: "The OSX Window Server crashes when all GPUs are used to the maximum"

Re:PyTorch on Apple M1 MAX GPUs with SHARK – faster than Ten... by noxa on Tuesday February 22, 2022 @06:34PM (#30435398)
Correct! We're currently compiling to SPIR-V and then going through MoltenVK/spirv-cross to get to Metal - and our SPIR-V needs quite a bit of work (currently tuned for Mali, which is a very different architecture). The biggest limiter though is as you note Apple doesn't have a way to use the fancy hardware instructions whereas our CUDA version is using the Tensor Cores.

The good news (for us) is that this is all effectively -O1 today; there's still potential for 2-8x speedups over these numbers and that's not even factoring in the inaccessible HW features that maybe one day Apple will expose :crossed-fingers: :)

// IREE dev

Re:PyTorch on Apple M1 MAX GPUs with SHARK – faster than Ten... by AceJohnny2 on Wednesday February 23, 2022 @01:35AM (#30438359)
Any chance of using the Neural Engine (otherwise exposed by CoreML) ?
Re:PyTorch on Apple M1 MAX GPUs with SHARK – faster than Ten... by magic_at_nodai on Wednesday February 23, 2022 @06:20PM (#30448056)
unlikely since the interface from ANE is not public and it may change between hardware versions.