News for hackers. Stuff that matters.
Re:Running PyTorch on the M1 GPU by danieldk on Saturday May 21, 2022 @08:04AM (#31457147)
I think in this case the PyTorch team also involved. They have already fixed some annoying bugs in the last two days (like matrix multiplication often failing because of buffer size mismatches).

As for BERT inference performance, I am not sure what kind of speedups you are expecting. The M1 Pro gives me 2.6 TFLOPs in single precision matrix multiplication of 768x768 matrices. M1 Pro GPU performance is supposed to be 5.3 TFLOPS (not sure, I haven’t benchmarked it).

Re:Running PyTorch on the M1 GPU by machinekob on Saturday May 21, 2022 @08:28AM (#31457279)
Ahh nvm I was thinking about m1 max (my brain is damaged by the m1 naming and i didnt saw information that you are using m1 pro gpu)
Re:Running PyTorch on the M1 GPU by danieldk on Saturday May 21, 2022 @08:36AM (#31457326)
Right, the Max should make a much bigger difference, since it has the same number of AMX units as the Pro, but double the GPU cores.