I expect either GPS guided rockets don't spin, or there is non-spinning part of the rocket that allows a GPS antenna to always point up.

Even inertial guidance would sound difficult with a spinning rocket, although I guess if you know exactly your rotational position relative to the earth, you could still calculate out how much you've moved up/down/left/right.