I think you make the strongest case for government investment (which is) in expensive, capital-intensive, high-risk projects; this is probably the most defensible area of government spending. There are two strong counter-arguments I have heard against yours. The first is that the reason private organizations do not invest in these projects is that the government's tax structure makes such ventures unprofitable, and the government crowds-out private investment in these projects. The second argument is that the government is the only investor in these projects because they are investing large sums of money when it is unwise to do so, because the project is not yet worth doing, or it is too risky, and a private organization would have invested less initially, and followed-on as the project became more affordable.
This cuts both ways - that's exactly why the government can and does shove money into things like infrastructure, basic research and, well, space exploration, while the private sector doesn't. There is no direct short-term profit motive for private sector in such endeavours. I emphasize "short-term", because the private sector is structured to care mostly about that. It takes a company to grow really big to invest in stuff like basic research - but at this point the internal structure of such company is more like that of a government than a free market.