Don't be afraid to break stuff yeah, as long as breaking something is a deliberate decision. Many times the root source of your technical debt is the design of your public API, it might be so bad it's hard even to create a wrapper for it. In order to ever get rid of that debt you must remove the entry point to it which means unhappy customers, make sure you have a new api ready and a transitioning path before depreciating the old one.

Unfortunately big test suites won't help for this type of refactoring though, on the contrary they might even be in your way.