Yeah, compounding interest is part of the metaphor. As long as you grow faster than the compounding interests, you are good. If the options for a startup is the keep growing or die, then taking on technical debt is reasonable.

Of course it is different for a steady-state company or organization. You need to keep technical dept at a manageable level.