I had a longer response written out, but my phone browser crashed and I have to go soon. This will be brief.

> externalized costs

Coase theorem and strong trespass law.

> Large scale goals with large public benefits but which come at high costs

Assurance contracts.

> public education system,

Private schools perform better across the board. If public schools didn't exist, I would expect to see more parochial and charity schools.

> public highway system

Replacing the IH system with privately operated toll roads is well within the reach of US investment banks, and it would be a safe bet. Plus they would be incentivized to actually do repairs quickly, unlike on public roads. Wins all around!

> public energy system,

Sure, this is a hard good to privatize due to line easement requirements. There are ways around this, but I think practically libertarians would agree with the government managing things like water pipes or electrical lines. This will become less of an issue in the near future as being off-grid on solar becomes practical for more people. (Or if we end up with small-scale community nuclear, like some are working towards.)

> Yet every market participant benefits enough that they are happy they exist.

If you think they'll still be happy after being made aware of how much they're paying (instead of it being obfuscated via taxes and respending) then there's no problem.

> armies

PMCs and military insurance. This would almost certainly be more efficient than the current bloated military we have in the US.

> disasters/disease

Insurance again.

> Would microsoft or walmart vaccinate everyone?

Does the government vaccinate everyone now? (No, they do not.)