Re:PyTorch 2.0 Release
by sebzim4500 on Wednesday March 15, 2023 @05:20PM
(#35174881)
Based on George Hotz's testing it is very broken. It's possible it has improved since then, I guess but he streamed this a few weeks ago.
[ Parent
| on HN ]
We tested inference for all spaCy transformer models and they work:
https://explosion.ai/blog/metal-performance-shaders
It depends very much on the ops that your model is using.
It supports a subset of the operators (as mentioned in the release notes). I don’t think it’s broken for the ones that it does support though.
Yes, this is my experience. Many off the shelf models still don't work, but several of my own models work great as long as they don't use unsupported operators.
Where can I find a list of the supported operators?
This is the master tracking list for MPS operator support: https://github.com/pytorch/pytorch/issues/77764
Re:PyTorch 2.0 Release
by mochomocha on Wednesday March 15, 2023 @05:56PM
(#35175279)
That's been my experience. However when fallback to CPU happens, it sometimes end up making a specific graph execution slower. But that's explicitly mentioned by the warning and pretty much expected.
Re:PyTorch 2.0 Release
by brucethemoose2 on Wednesday March 15, 2023 @05:59PM
(#35175317)
You'd think it would fall back to GPU/CPU for unsupported operations instead of failing, but I guess thats easier said than done.
In fact there is an environment variable that enables exactly that. There are obviously performance issues associated with that, but it does work.
Re:PyTorch 2.0 Release
by bigbillheck on Wednesday March 15, 2023 @05:39PM
(#35175099)
Based on George Hotz's performance at twitter I wouldn't bet he wasn't holding it wrong.
so, you would bet he was holding it wrong?
More likely, no bet would be placed.
Re:PyTorch 2.0 Release
by laichzeit0 on Thursday March 16, 2023 @07:47AM
(#35180724)
I could fine-tune a Detectron2 model a few months ago using PyTorch and MPS backend [1]. I'd be interested if it's working yet.