My experience says that you might not need much design for a typical CRUD app, but try to write a JVM/compiler/database and you will quickly see that a bad design pretty much aborts the given project and you have to start from almost scratch.

There is no incrementel rewrite between different stack/heap handling as those are an absolutely central parts of the design, which are pretty much impossible to try to encapsulate, as opposed to the 34th API endpoint. So what it means is that certain domains have much higher essential complexity and at that point the average encapsulation given by OOP/language tools are not sufficient to contain these parts, complexity will triumph and the whole program has to be viewed as one unified whole. Concurrent applications are a similar can of worms.

All the technologies you mentioned do undergo large component rewrites and refactoring very often. It's true that e.g. the JVM is sometimes hemmed in by decisions from the past. But it is a decades old project and it is not clear that more up front design and deliberation would have future proofed the project for the language and VM conventions of the 2020s.

I have applied incremental design to concurrent applications and a compiler + stack VM project that runs in embedded environments. You don't go in blind. You do need domain experience and broad strokes knowledge of the conventions. You make some major architectural decisions up front but these don't involve much planning or design. Contrary to your point about CRUD apps, API design is harder to achieve incrementally since it is an interface and requires cross-team (sometimes cross-organizational) iteration. It's still possible, but your organization needs to be equipped for incremental/agile work.

Yeah I mean for sure it depends on the domain.

But it's interesting you mentioned compilers - I'm in the process of writing one, and I very much used an incremental approach.

The first pass I essentially wrote a parser and a component which walked the AST and produced output. At a certain point it was clear that local knowledge of the AST wasn't sufficient to capture non-local details about the program which were required to produce the correct output. I got away for a short time with dirty tricks, but eventually transitioned to a new design: I kept the lexer and parser, and implemented a data driven IR in the style of an ECS system to be built up before emitting output.

So I threw out the initial output component, but I learned a ton by starting with an end-to-end compiler, however incomplete. If I hadn't taken that step, and tried first to plan the perfect IR on paper, I am certain I would have reached an inferior result.

edit: and even the IR and compiler middleware is loosely coupled. The IR is essentially a set of flat data tables, each of which is built independently by walking the AST. And the compiler is implemented in a series of independent passes: i.e. one pass to build the IR, one pass to derive type information etc. so it's very much grown into a series of independent components, each of which could be independently rewritten without affecting the others very much.

If you have infinite time to recover, there is no problem, but you could also design something perfect using that infinite time.

A bit of thinking about design and architecture can save you a lot of time. Start with the wrong data structures and maybe you'll have to patch a lot of thing or just redesign everything from scratch.

Be an architecture astronaut and you may never release whatever you're suppose to develop.

It's all about trade-offs.