News for hackers. Stuff that matters.
Re:My First Year as a Freelance AI Engineer by m0zg on Saturday May 02, 2020 @10:11PM (#23056344)
1.5.0 is basically broken for computer vision. Input tensors are usually in NHWC memory format, but PyTorch (and CUDA) prefers NCHW (planar). So you'd normally run permute() to move things around. But https://github.com/pytorch/pytorch/issues/37142 (and possibly other bugs) kinda gets in the way of that. I had to roll back, since training got way slower than it was before, and it wasn't super fast to begin with, even on my quad-GPU workstation.

The fact that such obvious, severe bugs make it through the release process likely means that there isn't really much of a release process. And what's in place doesn't even test the release on totally bread-and-butter models like resnet50.

Re:My First Year as a Freelance AI Engineer by smhx on Sunday May 03, 2020 @02:11PM (#23060795)
PyTorch maintainer here: we're looking into that and if needed will issue 1.5.1 asap. It didn't show up in release testing, which among other things does end-to-end imagenet runs with ResNet50 and a few other models (i.e. time and memory didn't regress). Will also figure out how to catch this early.
Re:My First Year as a Freelance AI Engineer by smhx on Sunday May 03, 2020 @08:30PM (#23063139)
@m0zg if you could comment on the issue you quoted with any details, it would be really helpful to us. Unlike what is reported in the github issue, `permute` isn't the regression.

For reference, one of the core devs added more details based on where we are with our investigation: https://github.com/pytorch/pytorch/issues/37142#issuecomment...