Major infrastucture and other construction projects are behind schedule and over budget constantly. So I guess I agree that software isn't that special, but I'm a different direction.

We are special in that nobody wants to spend the time to spec software in detail up front and we usually make major changes to requirements in the middle of the process.

If we had something like a detailed blueprint before writing a line of code and the requirements were set in stone we could get a lot closer to other engineering disciplines in terms of predictability.

With the kinds of planning and estimating processes we do actually use you’re lucky to get within a factor of two of reality.

The way I've heard it phrased is "If the physical properties of concrete changed every two years, civil engineering would look a lot different." (forget the source)

I think we're outliers in a couple of ways. One is what you mention -- we're usually both the architects and the construction crews of our projects, and we do both parts at the same time.

But there's also a step back -- how long does it take you to fully design, but not program, a particular piece of software? If you give a general contractor a particular set of blueprints, he can tell you roughly how long it will take to build, both the ideal number (if there are no scheduling or shipping delays) and what it probably will end up being. But if you tell an architect you want a design for a particular size and purpose and style of building, she can also tell you roughly how long it will take to draw that up.

How long does it take to design a particular piece of software? What makes a design take a day or a month to iron out, well enough that you can get a correct estimate of the amount of time it will take to build it?

I don't think we're good at that, as a profession.

> I don't think we're good at that, as a profession.

We aren't. And, as an industry, we have managed to convince ourselves that's a good thing. Agile methodologies, after all, are an attempt at making such efforts unimportant.

Yeah, we are not special. Estimation doesn't scale for any other kind engineering either.

true, but the comparison to other engineering disciplines does not hold up well at all: while everybody understands that planning when building a House is important as once it stands you can only change it at significant cost this does not hold true to software development. here we press a button and the house is constructed and with CD even delivered to the customer. my take: we ARE very special as our problems have not ever be encountered in the course of mankind due to the very nature of software.

It’s hard because a good portion of tasks which software developers need to estimate are not iid hence the law of large numbers doesn’t apply.

Nope, that blogspam article misses the mark by a mile. The major difference in almost all software environments is that the software requirements constantly change, even after the initial version is done.

Estimates in civil engineering are going to look just as bad as software if it was common to keep adding floors of different sizes to a building after it was complete followed by a “pivot” of turning the building into a support structure for a suspension bridge.

Software that is as rigid as tradition engineering projects in the real world can certainly be estimated, this happens in safety critical stuff all of the time. But the estimates are long, and requirements cannot change without massive delays, which is terrible for most software use cases.

You realize that this sort of requirements change happens all the time in construction projects, right? Granted there are limits to this, but the vast majority of construction overruns are caused by changes in requirements.

Requirements change in non-software projects but rarely do they change the project into a completely different category like "I know we started building an office tower and it's halfway done, but now the client wants it to be an airport." This happens in software all the time.

Do you know why construction projects don't get huge last minute changes very much? Because every change request is a billable event.

Software companies don't do this much. I've worked at only one that charged for changes, and -- surprise -- such changes were very rare.

Yup, in fact this is the strategy for profit maximisation for many, many construction companies. You look at the plans/requirements and if you spot loads of missing things you bid a low price with high prices for changes and make loads and loads of money.

Source: this is what my Dad did for a living, and I spent a couple of years doing it in my twenties.

It is also the strategy for profit maximisation for many, many software companies. "Change request" is a big deal in outsourced enterprise software dev. It is so bad that 'honest' software development groups that estimate accurately and don't have a culture of change requests cannot compete.

It’s still a change before it’s done and it’s pretty constrained. Scope changes too big are just completely impossible because of physical constraints, lack of permits, etc.

The requirements changes in construction projects are child’s play compared to the ridiculous stuff in software. “Oh, now that we have this data visualization app finished, please add in support for collaborative visualization exploration with synchronized views and voice chat.”