I refactor code with dynamic types all of the time with no problems -- but the code I work on has a lot of tests (or I'll add them), and I definitely never refactor code that lacks tests, whether it has dynamic or static types.

It's almost as if different people have different working styles that make different aspects of a language useful.