At scale, R&D can, to a limited extent, be commoditized. Companies/organizations (think particle-accelerators, etc.) have substantial experience bringing new technologies into production. If a new process has been proven in the lab and estimates exist for the speed with which each technology-demonstration milestone can be reached, one can credibly estimate when a new technology can reach the market. It is an inexact science, but all science is inexact.
With sufficiently accurate estimates and conservative allowances for uncertainty and schedule-slippage, one can reliably do this. Intel did so for decades, using Moore's Law as a roadmap.
But, as you suggest, the same general patterns apply to a lot of things. You know what you want to do. You have some idea of what you need to do to get there and how hard it's going to be. So people who have experience in such matters can make (usually) reasonable estimates of how quickly they can get there--even if they're (also usually) often optimistic. Because you're more likely to run into hiccups than unexpected breakthroughs.
Chipmakers don't always get it right. Past chip shortages have generally been transitory, and the punishment for overzealous buildout in response to those shortages would have been a ton of spent capex with a long recovery duration. Neither equity investors nor company executives like to lay out the kind of money needed for a new fab because it is a risk today for a reward that may or may not present itself years from today.
Chipmakers don't know the demand picture in the future, and they don't always execute the roadmap as predicted. If they did, we wouldn't have shortages today, and Intel wouldn't have delayed 10nm for so many years.
Maybe I'm not understanding your question, but it's certainly not dumb.
> If they know they’ll be able to do still finer etching in 4 years, what’s stopping them from doing it now?
They can't get the machines, for one. There's a substantial lead time for an EUV machine and ASML only keeps enough inventory to smooth out its delivery timeline. These are gigantic machines and they require much labor and expense to ship and install.
Also, remember that not every chip attempted is a chip made. As years pass, research and experience allow companies to get a higher yield out of a particular process. Higher yield means less cost per unit and more units produced per day. They won't roll out a product that is uneconomical to fab, and they won't roll out a product for which there is no chance in hell of meeting customer demand due to low yield.
> How can you not be able to do something today, but still know fairly accurately you’ll be able to do it in future?
Semiconductor hardware is one of the most high-inertia spaces in the technology sector. It's totally different from the startup world where you can build a company on little more than colorful tank-tops, ramen, and a pad of foam in your friend's dorm room. Semiconductor fabs take years to build, and the power and water infrastructure that underpins them is essential for them to function. This was the case two decades ago when Katherine Derbyshire wrote this[0] article, and it's even more the case now.
[0] https://www.dpr.com/assets/news/2002-06-01-semiconducotr-mag...
When you move to suv, for example, you need finer etching and better cleans and many other things. Until you introduce Euclid if you don’t need the finer etch and your process “yields” well don’t fix it if it ain’t broke.
An analogy would be…star drive screws are better overall than philips screws. But I probably wouldn’t switch my car assembly line over to star drive screws by doing it one assembly station at a time.
I’d shut the line down for a weekend and change them all, or I might change them all as part of switching production over to next years model.
A single vendor couldn't speed this process by much, because he won't have those missing tools.