In the enlightenment you could make discoveries about lightning with a kite and a key, in your free time when you weren't working as a publisher or founding America.

In 2020 you need 10,000 scientists and $9bn to build a hadron collider, and 99.9% of people won't even understand if your result is important or not.

Are we less good at science now, given that we're spending far more resources for far less impact? Or rather, are we better, but the lowest hanging and juiciest fruit got picked first?

> In the enlightenment you could make discoveries about lightning with a kite and a key, in your free time when you weren't working as a publisher or founding America.

And yet those discoveries weren't made in the previous thousands of years during which people had been standing around in thunderstorms. The enlightenment represented a paradigm shift where a new innovation - the scientific method - was applied en masse for the first time and it led to revolutions in science, technology, and society. Whenever a paradigm shift happens, be it the practical steam engine or long range electricity transmission, you invariably have a lot of suddenly accessible low hanging fruit. The mid 20th century had a surprisingly large number of such paradigm shifting innovations such as solid state transistors, integrated circuits, jet engines, nuclear reactors, antibiotics, solar cells, the internet, lasers, credit cards, the list goes on. Entire new fields of science and sectors of industry popped up in a very short period of time.

We continue to innovate, but our innovations are mostly evolutionary and there have been comparatively few paradigm altering inventions in the past 30 years, especially when one considers how much larger the population of sufficiently educated and affluent individuals who could produce such innovations has become and how much easier it is for such innovations to be communicated. It is impressive how good we've gotten at picking higher fruit, but we should still be asking why we are struggling so hard to find low hanging fruit.

The big problems in science can be solved by much smaller teams. LHC is just proving out the dominant paradigm. It’s not gonna help us understand dark matter and _that’s_ the real anomaly in the current physical understanding of the world. Crack that an you’re the next Einstein/newton or at least Planck/schrodinger/hooke.

There’s plenty of room for innovation in green energy tech, housing materials, ag science, etc... We’re objectively spending more to get those than we used to need to. And that’s WITH google and scihub! Something is rotten.

You seem to have a fundamental misunderstanding of how scientific research (especially physics) works. You take a model. You figure out some observable predictions of that model. You run an experiment to test that prediction. If it's right, the experiment stands as support to the model's accuracy. More importantly, sometimes the predictions are wrong. After a pattern of anomalies, someone comes up with a model that explains them. Rinse, repeat.

The idea that the "big problems" can be solved by small teams pontificating is missing all but that last step.

We must be talking past each other, as I understand it quite well.

My point (which I understand is controversial) is there's no requirement for billions of dollars and thousands of scientists to make the next big breakthrough. There's still low hanging fruit. Less than there used to be, but it's still there.

The LHC and big $$ funding is _one_ way to do it. So far its been unable to solve the biggest problems we have in physics (agreed that it's helped disprove some).

We're at a point now that is similar to what people were saying in 1900. Some people say "we've almost dusted this physics thing off and figured it all out". I'm fairly confident something is going to come along from some small corner of physics that will knock all our socks off the same way Quantum Mech and Relativity did... it may arrive this year or it may arrive in 120 years more to get there though.

The big problems in science can be solved by much smaller teams is my point. Sure it will take an army to completely work out the implications of the breakthrough. But the breakthrough "aha" can still come from an individual or small team.

> It’s not gonna help us understand dark matter

In case anyone wants a quick link to the LHC dark matter program, check the ATLAS [1] and CMS [2] summaries.

TL;DR: The LHC has ruled out a good chunk of a few popular dark matter variants already. There are a lot more searches in the pipeline. They haven't found anything yet, but of course nature doesn't just drop particles in the first place you look.

There are also some crazier plans in the works [3] which use the existing beams to look in other places.

[1]: https://atlas.web.cern.ch/Atlas/GROUPS/PHYSICS/CombinedSumma...

[2]: https://twiki.cern.ch/twiki/bin/view/CMSPublic/SummaryPlotsE...

[3]: https://en.wikipedia.org/wiki/FASER_experiment

> We’re objectively spending more to get those than we used to need to. And that’s WITH google and scihub! Something is rotten.

What’s rotten exactly? Take a look at this lovely graph [0] showing the cost of solar cells going from $76.67/watt in 1977, to $0.74/watt in 2013 (the price of installation is obviously more than the cell production costs, at about $2.5-$3.5/watt today).

[0]: https://cleantechnica.com/2014/09/04/solar-panel-cost-trends...

Surely you see the difference between explaining what dark matter is, and making solar cells cheaper.

Which one fits your definition of important scientific question that we should answer?

The years 1945-1988 are exactly the years of the Cold War as well. It seems likely to me that much of our innovation did not come from corporate research labs alone, but from government and defense spending. And defense spending was not just bullets and band-aids, but also basic research. (My favorite example is the Lambda the Ultimate papers were paid for by the Office of Naval Research.)

It's not surprising that the period that covers the invention of transistors, integrated circuits and microprocessors saw significant technological leaps. You can make the same observations about the periods when steam, gas and electric power came to market.

What you're describing though is a small number of companies making significant technological leaps, not the quantity of innovation. People also argue that the invention of the internet, and the subsequent global proliferation high speed internet represents the same sort of revolutionary leap forward (one that we're going through right now).

people argue that but it's not anywhere in the numbers (and even more importantly, in lived experience). The argument can be made that this innovation just lags the invention of the internet but that's kind of unfalsifiable and looks increasingly like rationalisation.

One key difference is that inventions like the microprocessor or the steam engine are deeply fundamental. They're at the 'backend' of the chain. Internet based digital services are at the consumer end. Which has produced some growth, but could be argued to mostly fuel hedonistic consumption. (It's hard to figure out what deep growth watching hours of tiktok creates)

In his book Gordon points to the thought experiment of going to sleep in 1870 and waking up in 1970 in NY. You'd be in an entirely transformed world. Cars, buildings reaching into the sky everywhere, electrified subway stations, computers, drugs that save countless of millions of lives, modern agriculture and so forth. Go to sleep in 1970 and wake up in 2020 and what's changed, other than people staring at tiny screens?

But you're not describing the pace of innovation at all. You're just describing your own judgements about the value of modern innovations. What would COVID-19 look like in 1970? I bet there wouldn't be many people working from home...

COVID-1970 would be a lot slower in the spread due to much less globalized world. The West probably would still not have experienced it, neither the Soviet Bloc due to then fresh Sino-Soviet split and heavily militarized border.

I think the Spanish flu might not agree with that assessment. Both plagues also did a pretty good job of spreading to pretty much all territories that had any form of contact at the time.

IMO the better example was 1957 Asian flu pandemic. It also started in China in December-February, but was not yet registered in the West as of May.

It also never crossed the Iron Curtain into Warsaw Pact countries.

2070 very probably will be entirely different than 1970, though. What changed is that computers have allowed us to go to space (at scale) and other incredible feats.

I remember thinking in 1980s that year 2000 would be something unrecognizable. You can feel this optimism in movies like Back to the Future, Blade Runner, Terminator or Escape From New York. They would set entirely different worlds in relatively near future, largely due to expectations from then recent technological change rates.

In reality (aside from tremendous change of political landscape) the 80s and the 00s weren't particularly different to live in.

The ability to have all information at your fingertips everywhere is a big big change.

The choice of New York drives some of this. Go to sleep in Shanghai in 1970, and wake up in 2020...

You're focusing on semiconductors but the same period saw a large number of new plastics, velcro, contact lenses, credit cards, the atomic clock, the orbital rocket / artificial satellite, the laser, the seat belt, fibre optics, the LED, and many other materials science breakthroughs.