The NIH budget is about $40B, 90 percent of which goes to research via about 50,000 extramural grants and the NIH’s own intramural research program. Very little of this is expected to “work” in the sense that it produces an immediate improvement in human health. Same goes for the NSF. The DoE, NASA, and DoD also fund plenty of research beyond the ARPAs (DARPA, IARPA, HSARPA, etc).
The ARPA programs are outliers in that they’re more likely to take larger risks in exchange for bigger rewards, while the NIH (etc) are more incremental. Nevertheless, it’s not true that these programs try to avoid ‘failures’ at all costs.
This does not mean the VC will actually be better at the task (producing better basic research results), but that it is more realistically going to get the funding in the current climate.
If you look at what's worked for computing in the past -- where the real qualitative changes came from, in terms of basic research -- you have ARPA, PARC, and Bell Labs. They all had the following in common:
- Long-term time horizons of 5-10 years minimum; - Lots of funding with few strings attached - Funded people, not projects - Operated in a given context, but goals were vague (enough freedom to play)
Any funding model that is sensitive to the pressures of quarterly earnings reports is not going to be able to do any of this. Which is precisely why most of the improvements in computing since the early 80s have come as side effects from quantitative improvements like speed, memory, cost, etc.
The current computing culture, and especially the funding model behind it, will not give us the big new inventions we hope for.
NASA is the obvious next example. They pay for a lot of esoteric stuff that once made they don't actually end up needing - but having paid the startup costs - end up contributing to other fields.
I think FEMA also does similar things though at a much less parallel (but a higher volume).
Most of the "regulate and promote an industry" (a dumb idea for structuring them but I digress) agencies also do this. The NRC for example does a lot of research on storing and recycling nuclear waste, but also on new nuclear technologies.
And other military agencies aren't worth leaving out. The army core of engineers pushes a lot of contractors to deliver unique solutions that end up getting resold, see the concrete barriers in Iraq.
NIH and CDMRP grant programs is several additional billion.
VC backed rockets have already taken longer to take a human to orbit than government money took to take a human to the moon. This is on top of the fact that they have had 60 years of technological improvement.
Simply put VC and capitalism in general is great for giving you turds rolled in glitter, less for building the sewage system to flush the turds out.
We still haven't seen any of the space startups match a manned Mercury-Redsone, which took 3 years.
During the Mercury program Nasa was using on average 0.4% of the federal budget, which was little over 2 billion a year in 2020 dollars.
The big spending was on the Apollo missions which are pipe dreams for any of the current space startups 15 years after their foundation.
I dont think it has to do with startups vs not.
Politics does not work well with "most of these projects failed". It's too easy of a target. Anything that failed, is an easy target for a congressman to take aim at as being "wasted", because in some sense that particular investment was wasted, in the same sense that every insurance payment is wasted except for the one right before you need insurance.
But, the logic behind "many long-shot bets in parallel" is inherently more difficult to explain in the political arena, which is why I think VC may actually be better at some of these kinds of things.