I also do back pocket when walking, swap pockets when sitting down. Do you also worry that one day forgetting to switch will get you into a lot of trouble? :)
I've got a single short that does that (re: bunching feeling) and it's just because the pocket isn't deep enough for the phone to sit flat against my thigh.
I'd rather choose my shorts more carefully in the future than switch to a smaller phone (that might still have the same problem if not tiny) just because of that.
This wasn't an issue when phones had 4.7"-5"-inch screens. Nowadays the phone goes into the cargo pocket if I'm wearing shorts, or back pocket if I'm walking.