Just be aware that it can be easy to get lost in a yak-shaving exercise doing your cleanup refactoring. I recommend timeboxing this phase to no more than an hour, or prioritizing one or two specific tasks so that you don't go off the deep end.
Don't forget that others are depending on you to get their work done. There may be a 2-hour cleanup task somewhere else in the code that will help your teammates far more than the itch you want to scratch here and now.
When I read comments like this I think to myself "How can they possibly be so productive that refactoring can be done inside of an hour?".
Then I remember that some developers split up projects/problems into really small components and can complete a feature or two in a single day. Well then an hour is actually a fair bit of time to spend refactoring.
"Convert your working code to good design" is not a trivial task, and like you say, a big part of it is to pick your battles.
The stake holder should be your customer, not your boss. Even if that dynamic is broken, you don't need to tell them it's shippable until you're happy with the code.