Re:PyTorch 2.0 Release
by simonw on Wednesday March 15, 2023 @05:14PM
(#35174816)
"the MPS backend" - that's the thing that lets Torch run accelerated on M1/M2 Macs!
[ Parent
| on HN ]
Re:PyTorch 2.0 Release
by sebzim4500 on Wednesday March 15, 2023 @05:20PM
(#35174881)
Based on George Hotz's testing it is very broken. It's possible it has improved since then, I guess but he streamed this a few weeks ago.
We tested inference for all spaCy transformer models and they work:
https://explosion.ai/blog/metal-performance-shaders
It depends very much on the ops that your model is using.
It supports a subset of the operators (as mentioned in the release notes). I don’t think it’s broken for the ones that it does support though.
Yes, this is my experience. Many off the shelf models still don't work, but several of my own models work great as long as they don't use unsupported operators.
Where can I find a list of the supported operators?
This is the master tracking list for MPS operator support: https://github.com/pytorch/pytorch/issues/77764
Re:PyTorch 2.0 Release
by mochomocha on Wednesday March 15, 2023 @05:56PM
(#35175279)
That's been my experience. However when fallback to CPU happens, it sometimes end up making a specific graph execution slower. But that's explicitly mentioned by the warning and pretty much expected.
Re:PyTorch 2.0 Release
by brucethemoose2 on Wednesday March 15, 2023 @05:59PM
(#35175317)
You'd think it would fall back to GPU/CPU for unsupported operations instead of failing, but I guess thats easier said than done.
In fact there is an environment variable that enables exactly that. There are obviously performance issues associated with that, but it does work.
Re:PyTorch 2.0 Release
by bigbillheck on Wednesday March 15, 2023 @05:39PM
(#35175099)
Based on George Hotz's performance at twitter I wouldn't bet he wasn't holding it wrong.
so, you would bet he was holding it wrong?
More likely, no bet would be placed.
Re:PyTorch 2.0 Release
by laichzeit0 on Thursday March 16, 2023 @07:47AM
(#35180724)
I could fine-tune a Detectron2 model a few months ago using PyTorch and MPS backend [1]. I'd be interested if it's working yet.
Yes, I am not sure at what extent is MPS a viable alternative to CUDA. You seem to write a lot about ML models. Do you have a detailed write about this subject?
Great! Any up-to-date guide to get the latest PyTorch running on Macs? Do we still have to use conda for example?
Re:PyTorch 2.0 Release
by PedroBatista on Thursday March 16, 2023 @11:15AM
(#35183043)
MPS is like an artificial flavor, you get the general idea but not the nutrition.
Nvidia execs should light a candle and pray to all the Gods most "AI" stuff really just works with CUDA since it was coded with CUDA in mind.
That's why I reluctantly shell out quite a few bucks on a ridiculously overpriced Nvidia card.