And backing out of it when running on pypy, as that does not deprecate python 2 compatibility...

Sure. Pypy is a separate implementation, they only control CPython. That's a pretty normal arrangement - official moves on, other forks might backport fixes for longer or focus on stability or some other realm of performance or something.

Probably only on a recent pip version. Pip 10's dependency resolution doesn't like our requirements files (we have contradictory versions that work due to the order they are in the file), so we've mostly only gone up to pip 9.