I thought it obvious, but the first hypothesis from the OP:

I'm not convinced that the government can get a better rate

From the data:

Single payer system countries have far lower costs for similar if not better outcomes.

The other hypothesis:

I think the real reason in Japan might be that costs are low because there is more competition due to more private doctors

From the data:

Japan has dramatically lower costs than the US, while having a radically different system far more similar to 'Medicare for all'. The far more private doctors comment was I think addressing irrational concerns that private healthcare would somehow disappear under medicare for all (instead of just being a lot cheaper). Japan is the proof that doesn't have to be the case.