This is definitely sometimes not easy, and of course it'll get more difficult generally the more code there is or the more complex (usually meaning bad!) the design is, but I'd almost put it the other way round to your perspective - the types are like the signposts in this design evolution and discovery process. They don't make it easier to do the work of refactoring a design, but they do make it much easier to understand when I'm going in a good/bad direction and also when I have succeeded.
However TypeScript’s structural typing is much more flexible, because it is designed to describe the types of all the existing JavaScript code out there (via the DefinitelyTyped project).
This gives the best of both worlds: the freedom to express the architecture you need, plus the freedom to actually refactor an existing code base into that new architecture without introducing many bugs. Dynamically typed languages give you the former but not the latter.