I'm mostly talking about minor cleanup like finding the best names, removing duplication, deleting unused code. That can be done quickly, and is often easiest once the dust has cleared and the design is finished and working.

If the code is working but is badly designed, that can take longer.

Occasionally, the new feature requires larger refactorings to redesign preexisting code. That can take days, but is of course essential to do.