> The cost of letting Stripe do all the work than moving over to another provider is extremely costly and can damage sales.

Yes, that's exactly my point: At some point you are completely locked in with a single supplier that holds your entire income stream and perhaps even more than that.

Ideally (easier said that done, I know) you want to have at least 2 suppliers for any key piece of infrastructure as early as possible and to avoid letting a supplier 'expand' the number of tasks they do for you too much.

The latter seems to be Stripe's strategy: They start with payments then expand step by step in everything related et even in things like company incorporation.

I imagine that most businesses have bigger risks to deal with than this, especially in early to mid stages (where Stripe seems to target).

Perhaps. You should always keep that in mind, though, and avoid building your system tightly-coupled to any single supplier.