There are too many people in the space industry who can write a nice paper showing what they could do if only you gave them a billion dollars.
Obviously private space (beyond GEO) is going to be huge at some point. The problem is bootstrapping.
When you have a chicken and egg problem, the solution is to build some small cheap thing that works to demonstrate your competence and attract larger opportunities.
John Carmack's started his own space program on about $5M of his own money and 10 years of effort. How many people in the US buy homes for more than $5M? How many shitty web apps get more funding than that?
He has a device that can lift off the ground, move a couple hundred meters to the side, and land.
That's a long way from a space program.
There are a lot of subtleties in liquid-fuel rocket engines.
Simply getting them to ignite reliably and to re-start were seen as a Big Deal, only a few years ago.
I wouldn't worry about space spending, or space infrastructure when our AF is spending oodles to put weapons and craft into space.