Einstein didn't attend the American public school system, nor did Dyson, or many of the other great scientists of the early 20'th century. In the present day, Sergei Brin only spent 4 years in US public schools. Elon Musk didn't, nor did Max Levchin (Thiel's wikipedia page is ambiguous). Zuckerberg didn't go to a public high school either (don't know what he did before that).
You can't point to a country with massive contributions from immigrants, and then declare that services those immigrants didn't use (or used minimally) are the cause of prosperity. The logic just doesn't work.
It also ignores the armies of undergraduates and computers (the human kind, mostly public school educated women sitting at adding machines) that made most American discoveries and inventions possible. Individuals do make some leaps, but on the whole it's massive collaboration that advances science and technology.
I'm disputing the idea that the public school system made the US into a "20th century powerhouse". If you want to attribute all of a person's success to the school they pass through, be my guest. I can similarly say the IRS is the cause of the US being a "20'th century powerhouse" - after all, 100% of US businesses are taxed by the IRS.
But the fact is, the US is fairly unique in terms of innovation and growth. The education system is, by most measures I've seen, fairly mediocre [1]. And additionally, a fairly large amount of our workforce, particularly at the higher ends, didn't even use the public education system. It seems like an extraordinary claim to attribute the success of the US to our public schools. Yeah, we need public schools/indoor plumbing/electricity, but it would be similarly silly to claim we are a powerhouse as a result of those services.
[1] As far as I'm aware, this was always the case. I know that during the Cold War, we were deathly afraid of the Soviets with their high test scores gaining a scientific advantage over us.
PS: China and India both have huge public education systems.