There's a lot of discussion on this online and some folks speaking past each other. Yeah, there has been the invention of the internet, faster computers, the blue LED, sequencing the human genome...etc, but that is argued to mainly be engineering innovations on already understood physics. Where is the next discovery on the level of general relativity? Are we just at diminishing returns now where all the low hanging fruit has been found? How many physicists wasted away researching string theory? Were we just putting resources in the wrong place?
I do strongly think that modern research has become so beauracratic that it gets in the way of actual progress. The endless paperwork, presentations, teaching...etc isn't very conducive to discovery. Your average professor is more like a project manager than what Newton did.
See for yourself: https://m.youtube.com/watch?v=AWI70HXrbG0
If you want some true genius, behold the shear terror of the near field of a dipole antenna. All shall view her and despair.
https://courses.grainger.illinois.edu/ece350/ece350lecture_n...
You need a certain amount of motivation, grid, etc. If these skills would be broadly available and the situation also, we would have a lot more Einsteins today.
There is plenty of things which do not require global projects building multibillion devices.
Mathematic thinking for ML for example.
DeepSeek paper for example.
There isn't a shortage of smart people or opportunities to make technological progress. The handicap is powerful humans looking at new approaches and saying "No" with the firmness of a man who sees a threat to their position. Even today I have a list of things that from an engineering perspective would probably result in improved prosperity, it just isn't possible to get them through the political process (nuclear power springs to mind, but I'm sure there are less controversial examples - Uber v. the Taxi industry maybe).
The people on the bleeding edge of science and technology aren't just sitting in futuristic labs, surrounded by holograms and AI tech assistants, spouting one brilliant insight after another.
They are spending most of their days arranging various types of poorly formatted data, be it scientific metadata, purchase orders, journal proofs, grant proposals, interview schedules, their shopping list, or maybe (for a brief moment) actual scientific data. The real productive geniuses are the ones who figure out a way to ignore all this noise for along enough to get something done. It all comes down to logistics in the end.
A huge fraction of this work could conceivably be automated, or just trivial if people migrated to a better data structure. Maybe LLMs will help us bring order here, but that's only half the battle: part of learning to format data is thinking about the formal structure, what consumers will actually need, what parts are duplicated, etc.
20 years ago Tim Burners-Lee was advocating to clean this up via a better structured web [1], but, for various reasons, we haven't made much progress. I suspect a big reason is that the vast majority of people would struggle to understand the purpose of structured data: like basic literacy it's only useful if other people in your community use it.
For the alternative viewpoint: https://slatestarcodex.com/2019/02/26/rule-genius-in-not-out...
What really drives progress, I think is production and technology. Like better telescopes. For the cosmic distance ladder, the clever mathematical tricks are nice, and necessary, but the real important part is the precise measurements. And for that, we need precise instruments. And precise instruments need good quality materials, skilled tradesmen, good tools, etc... The tools themselves need the same, and the tools to make the tools, etc... A lot of the measurements involve travel and long distance communication, which wasn't trivial either. Mounting an expedition to the other side of the world requires a ship, a crew, etc... Then we need to collect these data, and people have to write these books, we need librarians, archivists, etc... All that so that finally, one guy can look at the data and do the maths.
And all that is far from the whole story. Skilled craftsmen and sailors need food and shelter, and the raw materials to practice their craft of course. They need farmers, miners, lumberjacks, etc... Farmers, miners and lumberjacks need their tools too. Advancing in the cosmic distance ladder is the work of millions of people over several lifespans, and some of them worked in cotton fields. We could wish the guy in the cotton fields was treated better, but it doesn't change the fact that in order to advance science, we actually need more people in cotton fields than people doing maths.
With technological progress, we can afford to have more people doing maths and provide them with enough data to be useful, but that's a very recent development.