1. Clearly define a vision for the future/ goal(s) that should be achieved
2. Get out of the fucking way of your minions, and trust that you hired correctly, to let them figure out how to get to the finish line
Bonus Points:
3. Your temperament is in the Goldilocks' zone of neither being too much of a spendthrift, nor too much of a miser, when setting budgets, i.e. you're not some rando without P&L experience that was tied to your bonuses.
Otherwise they’ll burnout and the resentment of the state of the codebase will be jet fuel to that brand new revolving door on your team.
Commiserate to some degree, ‘look, I know this shit sucks’. If they don’t know you empathize, they won’t trust you.
Are you spending your time second-guessing the developers and micro-managing them on the exact thing they are supposed to be experts?
Your phrasing has that "feel", but it's far from a sure conclusion. Anyway, if it is the case, you may need to reevaluate both your hiring (should you get more senior people?) and management (this is a clear "should never be done", unless your people are explicitly under training) practices.
That's a lot of trade offs to avoid maintaining the subset of features you need from said software in house, being able to leverage internal knowledge, being able to streamline all your environments and tooling to suit your needs instead of catering to the needs of said software.
This isn't an argument one way or another, it's just pointing out that there are trade offs you need to make consciously or otherwise.
In your position I would be extremely concerned about pushing off the shelf software onto Devs that either lack the clout to be comfortable pushing back or lack the communication skills to clearly articulate all the trade offs really taking place.
It's very easy for a boss figure to push through requirements with off the cuff pointed questions. The reports often need to push back with orders of magnitude more research and thoughtfulness than went into the question or suggestion.