I see people on here post construction projects as the poster child for things that can be estimated with traditional project management techniques. This is wrong because most construction projects come in over cost and over schedule.
This can be because materials costs change, labor costs change, the soil at your site may not be what you expected, weather impacts, and stakeholders may demand something different after the project is estimated.
Although these are unknown quantities, these factors existence are known. Whereas in software, you’re more likely to encounter unknown unknowns. These are factors that nobody thinks are even possibilities until you run into them because nobody has done anything similar before. An example from my experience is when I was working on implementing a marching cubes algorithm on a GPU, we did research to make sure that you could do it. But when it came time to implement, we found that the cube data was expressed as a set of faces, and not a set of vertexes and no efficient algorithm existed for that expression of the data that could be mapped to a single instruction multiple data paradigm. We could not preprocess the data to convert it to vertexes because it would not offer any speed up if we did. To implement the acceleration we wanted, you’d have to change how the data was stored in the application. Which is a very different project with very different timescales. We ended up abandoning the effort and moved to other work. This happened because we made a discovery that no one could have predicted, in the implementation, that had an unknown impact.
In physical projects, you might find there are a lot of unknown things that can happen, that you may not predict. You can certainly reduce this by bringing in experts to a project, people who have seen much more than you. Certainly there is still potential for black swan events. But these are rare. Where as in software, the experts you may bring on may not necessarily know more than you, you are frequently your own expert. Black swan events in software projects are common.
But they were surely predictable in aggregate, no? There's only so much volume in the house and a volume will take a bounded amount of time to deal with, even if the bound is very large. An oceanic vessel has a maximum amount of mass that can be added or removed, and every unit mass will take a bounded amount of time.
This isn't the case for development. There is a bound to how much code can possibly go in a project, but there is no bound for how much development work goes into any unit of code.
Only in the sense of things always take at least twice as long as they "ought" to. The volume of the vessel has very little to do with it. In a refit, you know some things going in--which may indeed be relatively straightforward (so long as the right parts arrive on time)--but for bigger jobs lots of things crop up including resource conflicts and parts that need to be ordered.
If software projects added more slack, they'd be more likely to be completed on time too. There's nothing magical about software. And, in fact, software doesn't need to deal with a lot of the quirks of physical systems.
To some degree and depends on the job. Start opening up walls in an old house and tell me how cookie cutter it is. When I used to run shipyard jobs on offshore drilling rigs, I can tell you that every single job ran into all sorts of problems that were individually not predictable.
It's also just nonsense to think that most software is this uniquely open-ended snowflake. Most engineering jobs require problem-solving that can be affected by all sorts of things in the physical world--plus you sometimes just have to build and try out designs and maybe they work, maybe they need to be refined a bit, and sometimes you have to toss them out and start again from scratch.