My wife works as an acoustical consultant at a global construction firm. The things you hear about factories, offices, and even hospitals is wild. Don’t get me wrong the construction world works very hard to avoid issues but I think we in software tend to hold other engineering disciplines up on a pedestal that doesn’t quite match the messiness of reality.

Thanks for saying this. I think we in software engineering tend to think too binary: either the product is perfect (100% bug-free) or it's shit. There's always room for improvement, but compared to other engineering, overall, I think we're doing pretty good. As an example similar to your wife's, my friend used to work for one of the major car manufacturers doing almost the exact same job as Edward Norton's character in Fight Club. The cars had "bugs", they knew about it, but they didn't publicly acknowledge it until they were forced to.

Trust me when I say this: even "other" engineering domains have to do patches.

The difference is that software can be used before it is fully ready, and it makes sense to do so. No one can really use a 90% finished power plant, but software at 95% capacity is still usually "good enough"

I think you're 90% there. There is also the cost to apply a patch.

If you want to patch a bridge, it's gonna cost you. Even if you only need to close down a single lane of traffic for a few hours you are looking at massive expenses for traffic control, coordination with transportation agencies, etc.

For most software it's pretty inexpensive to ship updates. If you're a SaaS company regular updates are just part of your business model. So the software is never actually done. We just keep patching and patching.

In some contexts, it is much more expensive to push out updates. For example, in the 00s, I worked on a project that had weather sensors installed in remote locations in various countries and the only way to get new software to them was via dial-up. And we were luck that that was even an option. Making international long distance calls to upload software patches over a 9600 baud connection is expensive. So we tested our code religiously before even considering an update, and we only pushed out the most direly needed patches.

Working on SaaS these days and the approach is "roll forward through bugs". It just makes more economic sense with the cost structures in this business.

Indeed. We calculate a $1 dollar fix in the factory costs $100 to fix on site.

e.g. product recalls?

I install high voltage switchgear on site. A common problem is all the changes that has been added during the design stage, circuits that have been removed or altered, work that has kind of mostly been done to the schemes by the overworked secondary engineer. Sometimes, the schemes have been changed after all the wiring is completed and shipped to site, making it my pain in the ass when it's time to do the commissioning.

The end result is never 100% perfect, but somewhere in between "not too bad" and "good enough".

Thanks for this insight! It has pretty strong explanatory power. It also explains why rushed development can stall. It explains 'move fast and break things'.

There's even an added factor of learning more about what is really needed by putting a 95% done product into use.

Heck, it explains (stretching it here) space-x's success with an iterative approach to rocket design.

There are a few aspects. One is that we don't understand the fundamentals of software as well as the underpinnings of other engineering disciplines.

More importantly though, for the most part we choose not to do engineering. By which I mean this - we know how to do this better, and we apply those techniques in areas where the consequences of failure are high. Aerospace, medical devices, etc.

It differs a bit industry to industry, but overall the lessons are the same. On the whole it a) looks a lot more like "typical" engineering than most software development and b) it is more expensive and slower.

Overall, we seem to have collectively decided we are fine with flakier software that delivers new and more complex things faster, except where errors tend to kill people or expensive machines without intending to.

The other contributing thing is it's typically vastly cheaper to fix software errors after the fact than, say, bridges.

The modern car contains within it a perfect example the dichotomy:

1. The ECU ("hard" engineering)

2. The infotainment system ("soft" engineering)

Now, an interesting thing I have noticed is that "soft" software engineering pays more. Often substantially more.

I think your salary observation is more of a firmware vs. hardware, rather then "soft" vs "hard" engineering.

Further to that, it's often informative to figure out what makes a company money. The highest paid software development roles tend to be doing things that are closer to revenue, on average. If you are a software developer at a hardware company (or an insurance company, or whatever), you aren't that close. Even worse if you are viewed as a cost center.

>Further to that, it's often informative to figure out what makes a company money. The highest paid software development roles tend to be doing things that are closer to revenue, on average.

yeah. Who are those trillion dollar businesses and what do they rely on?

- Apple: Probably the better example here since they focus a lot on user-facing value. But I'm sure they have their own deals, B2B market in certain industries, R&D, and ads to take into account

- Microsoft: a dominant software house in nearly every aspect of the industry. But I wager most of their money comes not from users but other businesses. Virtually every other companies uses Windows, Word, and those that don't may still use Azure for servers.

- Alphabet: ads. Need I say more? Users aren't the audience, they are the selling point to other companies.

- Amazon: a big user facing market, but again similar to Microsoft. The real money is b2b servers.

- Nvidia: Again, user facing products but the real selling point is to companies that need their hardware. In this case, a good 80% of general computing manufacturers.

- Meta: Ads ans selling user data once again

- Tesla: CEO politics aside, it's probably the 2nd best example. Split bewteen a user facing product that disrupted an industry and becoming a standard for fuel in the industry they disrupted. There's also some tangential products that shouldn't be underestimated, but overall a lot of value seems to come from serving the user.

General lesson here is that b2b and ads are the real money makers. if you're one level removed that financial value drops immensely (but not necessarily to infeasible levels, far from it).

> One is that we don't understand the fundamentals of software as well as the underpinnings of other engineering disciplines.

That sounds like an awfully bold claim. I have the feeling we understand software a lot better than we understand mechanical engineering (and by extension material sciences) or fluid dynamics. By a big margin.

I worked with finite element software and with CFD solvers, you wouldn't believe how hard it is to simulate a proper airflow over a simple airfoil and get the same results as in the wind tunnel.

> That sounds like an awfully bold claim.

To the contrary, it's nearly canonical. Most of the problems pointed out in the 70s (mythical man month) have still not been resolved, 50 years later.

>you wouldn't believe how hard it

Oh, I'd believe it (I've designed and built similar things, and had colleagues in CFD).

But you are definitely cherry picking here. The problem with CFD is we don't understand the fluid dynamics part very well; turbulence is a big unsolved problem still, though we have been generating better techniques. This is so true that in an undergraduate physics degree, there is usually a point where they say something like: "now that you think you know how lots of things work, let's introduce turbulence"

But a lot of mechanical engineering and the underlying physics and materials science is actually pretty well understood, to the degree that we can be much more predictive about the trade offs than typically is possible in software. Same goes for electrical, civil, and chem. Each of them have areas of fuzziness, but also a pretty solid core.

The article is about delivering a complete, working project "on time". I have a neighbor whose home is being renovated and it is already 2x the time the contractor originally quoted.

Of course it is easier for a developer to walk away from something incomplete than an architect and the contractors involved in a physical project, but still, I hardly think that there is really much difference in terms of timelines.

FWIW in my experience delays in e.g. home renos (or for that matter larger scale projects) are mostly for reasons unrelated to the engineering. In software projects, it's probably the #1 reason (i.e. we didn't know how to do it when we started).

Software is still absolutely king for number of large scale projects that just never ship, or ship but never work.

> To the contrary, it's nearly canonical. Most of the problems pointed out in the 70s (mythical man month) have still not been resolved, 50 years later.

Even with all of those applied, we wouldn’t be magically better. Complexity is simply unbounded. It’s almost impossible to reason about parallel code with shared mutable state.

Most engineering domains expect failure; the fail safes, checklists etc prevent it causing real damage.

Because other engineering domains are "actual" engineering domains. They didn't just co-opt the word to have fancier sounding job titles.

>when do not accept the same in other engineering domains?

No, you just complain that your taxes are being used to build expensive roads and bridges. Or you think airplanes are far too expensive. Or that new cars are insanely expensive.

There are cost trade offs. In general, better quality more expense.

Also in software there is not an excessive amount of software engineers in relation to demand for software. So SWEs can get paid a lot to go build crappy software.

We accept this in all fields of engineering. Everything is "good enough" and the seems to work reasonably well. You should remember this next time you hear about car recalls, maintenance work on bridges, or when some component in your laptop flakes out.

In addition to the other answers, there is the perennial and depressing one: Software bugs haven't killed enough people in a suitably visible/dramatic way to be regulated that heavily.

Imagine same approache in other domains:

Team are flying the airplane, the se time rebuild it to the zeppelin, testing new engines inflight.

Or construction. Let's build apartment block, but for few apartments we will test new materials, new layout, etc. Once there are walls of the first apartments we will let people live there. We will build how we can, according to the napkin plan. In the end we will put all tenants in and stress test strength of the structures. Or one day people return home and their apartments have totally different design and layout because someone from the HOA decided so to get a promotion.

I mean, bridges collapse. That hasn't meant we gave up on engineering bridges. Point being, we have some risk tolerance, even for civil engineering.

Now we don't accept an engineer saying, "this bridge will probably collapse without warning", which we do accept with software. So there is a difference.

Because complexity is boundless and in software it has no cost.

Building a house will have a restrictive initial budget for complexity, you don’t have enough in that budget for rotating floors, or an elevator that is catapulted to the correct floor, etc. These would cost both at engineering time and implementation time a huge amount. Less complexity is easier to analyze.

In case of software, complexity has negligible cost, relative to physical systems. You can increase it ad infinity — but proving it (the whole stack - from the hardware-OS-userspace software) correct is likely impossible with even the whole of mathematics, in certain cases.