Sure, but all you're doing is moving the uncertainty from the development to the planning phase, so instead of a development phase of unknown length and cost, you end up with a planning phase of unknown length and cost.

Waterfall gave the impression that planning and analysis is costly by insisting it should be carried down to the lowest levels of abstraction, before doing anything else.

The first involved taking all sorts of situational data from various sources in an airport, e.g. radar systems with their own binary formats, and combining them with the work of a postgrad researcher who had come up with an algorithm to optimise when pilots should turn on an aircraft's engine in order to save fuel and cut emissions.

The second involved managing 3G dongles on fleets of laptops, tracking and sending alerts on their usage and billing info in realtime, back in the days when business users would accidentally run up £10,000 bills abroad.

If at any point in the first half of either of these projects I'd been asked to come up with an estimate for the total amount of work, it would have been wildly inaccurate. Each project had both low and high level surprises at every turn that would have just compounded to blow those estimates up. There is basically no significant way in which one informed the other, even though parts of the systems used similar event sourcing architectures on the exact same server-side stack, and were done for the exact same startup which broadly offered real-time systems to utility companies of various sorts.

It's entirely possible that there exists a breed of programmer who could have estimated these projects in their entirety, much earlier and more accurately than we did, but I've never met them, and I'd rather work in places that didn't compel people to.

I've known and worked at other places where basically any software methodology would (and did) work, but I certainly don't think it's rare (as you seem to suggest) to find yourself at the other end of the spectrum.

Doing something about it takes some commitment, however. Nobody estimates well without having practiced it, and been honest with themselves about where it went wrong.

I disagree. Even when you're doing mundane everyday stuff you have done a thousand times before, one trivial variation may throw of your estimate completely. Maybe an edge-case in this specific project triggers a bug in a 3rd party's library, and you're losing weeks waiting for the 3rd party to roll out the fix (been there, done that). You just don't know in advance

> you can do analysis and planning at a much higher level of abstraction than development

Of course, but then you'll get an inaccurate estimate.

> Waterfall gave the impression that planning and analysis is costly by insisting it should be carried down to the lowest levels of abstraction

That's only necessary if you want an accurate estimate. I think accurate estimates are pipe dreams.

It's called spec programming, where you have programmers slinging code against precise specifications. Common in ERP and regulatory or business compliance-driven code.

On the other hand, you have development. That is where you're attacking the less defined problems. Sure, we know the tools in our toolset, how to apply them, and time it takes to apply each tool. Problem is, the time to figure out the series of tools needed and their precise applications burns enormous time, time that could have been spent towards a MVP.

In the use of the adjectives 'precise' and 'enormous', you are just repeating the same false dichotomy in a slightly different form, by ignoring (or not understanding) my point about the use of abstraction in analytical thinking.

The people who are most successful in what you distinguish as development understand a good deal more about what they are working on than spec programmers need to, and they are not, as you imply, groping around in the dark for a solution - that's what spec programmers do when faced with a development problem.