In software, a single bit-flip or typo can lead to catastrophic failure, it is sensitive to defects in a way physical systems are not. The behavior of software is dependent on the behavior of the hardware it runs on; unlike physical engineering, the hardware for software is not engineered to the specifications of the design, it has to run on unknown hardware with unknown constraints and work correctly the first time.
Physical systems are inherently resilient in the presence of many small defects, a property that physical engineering relies on greatly. Software is much less tolerant of defects, and there is a limited ability to "over-engineer" software to add safety margin, which is done all the time in physical engineering but is qualitatively more expensive in software engineering. Being broken is often a binary state in software.
I've written software for high-assurance environments. Every detail of the implementation must be perfect to a degree that would make engineers with even the most perfectionist tendencies blush. Physical engineering requires nothing like this degree of perfectionism. In my experience, the vast majority of engineers are not cognitively equipped to engineer to that standard, and in physical engineering they don't have to.
Totally agree with what you wrote but in any well run engineering organization one has to consider the cost/benefit of extra testing and engineering.
If you're making a USB powered LED desk lamp, you get into diminishing returns fairly quickly. If you're making basically anything on JWST, less so.
In software systems, correctness is binary, so they actually have to be defect-free. Defects don't manifest gradually and gracefully like they often do in physical systems.
Every detail of the implementation must be perfect to a degree that would make engineers with even the most perfectionist tendencies blush
the amount of software that fits this description is likely comparable to amount of physical engineering that requires the same perfectionism…
AWS is using a lot of Formal Verification and Automated theorem proving on their core systems like S3 and TLS to increase reliability.
That can happen in physical engineering too, if it's not done right. Likewise, if a single bit flip can lead to catastrophic failure, that's an indication that the system was poorly engineered.
The problem with software is not so much that it is fundamentally different in terms of its failure response (though there certainly are differences -- the laws of physics provide important constraints on hardware that are absent in software) but rather that it is so easy to copy, which tends to entrench early design decisions and make them difficult to go back and change because the cost is too high (e.g. syntactically-significant tabs in 'make'). But even that can and does happen in physical design as well. The Millennium Tower in San Francisco is the best example I can think of. The World Trade Center is another example of a design that failed because the failure mode that destroyed it was not part of the original design requirements. When the WTC was designed, no one imagined that someone would fly a jet into them some day. Today's adversarial environment was similarly hard to imagine before the Internet.
It absolutely is possible to engineer software for robustness, reliability, and trustworthiness. The limiting factor is economics, not engineering.
That's why you have tests for this.
The main problem that i see with software, is that the testing and bug finding part are left to the user.
And, when testing is done, it is done as easy as possible. For example a function must return 5 when it has 8 as an input. They only test for 5 as an input value and 2 or -1 are left to the user to test them.
No.
What you’re suggesting is about as plausible as the Aesop fable about mice saying they should put a bell on the cat. Sounds great, completely impossible.
So what about only new code then? In that case, does old code get grandfathered in? If so, Google gets to take tens of billions of lines with them for free, while startups face the audit burden which would be insurmountable to reach a similar scale. Heck, Google does not have enough skilled labor themselves to audit it all.
Also completely unfeasible.
And even if, even if, some country decided to audit all the code, and even if there was enough talent and labor in this world to get it done by the next decade, what does that mean?
It means all research, development, and investment just moves to China and other countries that don’t require it.
Also completely unfeasible.
> “Their livelihoods should be on the line.”
This fundamentally relies on the subject being so demonstrably knowable and predictable, that only someone guilty of negligence or malice could possibly make a mistake.
This absolutely does not apply to software development, and for the reasons above, probably never will. The moment such a requirement comes into existence, any software developer who isn’t suicidal abandons the field.
When something like that does happen, it is very hard to know the measure of responsibility every entity holds. This will most certainly be decided in court.
In computer code, every last line is possibly structural. It also only takes a single missing = in the 1.2 million line codebase to kill.
Comparing it to school engineering is an oversimplification. You should be comparing it to verifying the structural integrity of every skyscraper ever built; for each project.
Leaded paint, Aresenic, flammability, weight (steel walls vs sheet rock).
The complexity is still less than software though, and there are much better established standards of things that work together.
Even if a screw is slightly wrong, it can still work.
In software it's more like, every screw must have monocrystalline design, any grain boundary must be properly accounted for, and if not, that screw can take out an entire section of the building, possibly the whole thing, possibly the entire city.
There is a lot of literature on civil engineering failures. In fact, my civil engineering education was largely structured as a study of engineering failures. One of the most striking things about forensic analysis of civil engineering failures, a lesson the professors hammered on incessantly, is that they are almost always the result of really basic design flaws that every first-year civil engineering student can immediately recognize. There isn't some elaborate engineering discipline preventing failures in civil engineering. Exotic failures are exceedingly rare.
Outside of those rare cases, everyone is demonstrably unwilling to pay the unavoidable costs of providing these guarantees. The idea that software can be built to a high-assurance standard by regulatory fiat and everyone just gets to freeload on this investment is delusional but that is what is often suggested. Also, open source software could not meet that standard in most cases.
Furthermore, those guarantees can only exist for the narrow set of hardware targets and environments that can actually be validated and verified. No mixing and matching random hardware, firmware, and OS versions. You'll essentially end up with the Apple ecosystem, but for everything.
The vast majority of people who insist they want highly robust software neither write software to these standards nor are willing to pay for software written to these standards. It is a combination of revealed preferences and hypocrisy.
Never forget as well that it only takes a single cosmic ray to flip a bit. Even if you code perfectly, it can still fail, whether in this way or countless other black swans.
2.8 trillion lines of code aren’t going to rewrite themselves overnight. And as any software developer can tell you, a rewrite would almost certainly just make things worse.
Plus, the bulding codes are safety-focused and often don't cover things that most people would consider defects: for example a huge hole in the interior wall is OK (unless it breaks fire or energy efficiency codes)
In software, when people think it counts, they do too. The problem is not all people agree on “when it counts”
I don't suppose most doctors or civil engineers want to take liability either.
That's the problem, isn't it ? Nobody wants to assume liability. They all want just the profits.
Think about, if your car will come with a disclaimer like Microsoft's EULA.
Not all software is equivalent and there is plenty of code that gets treated with the precision you're asking for.
But the ability to ship out code to billions in the blink of an eye is both the strength and the weakness of modern software.
It allows few engineers to tackle very complex problems without the huge investment in man hours that more rigor would require.
This keeps costs down enabling many projects that wouldn't see the light of day if the NRE was equivalent to HW.
On the other hand, you get a lot of "preliminary" code in the world, for lack of a better word.
At the end of the day all engineering is about balancing trade-offs and there are no right answers, only wrong ones. :)
Like Crowdstrike ? Or Microsoft ?
> The stakes are now higher with data being so important and the advent of algorithms that affect people directly. From health insurance claims, to automated trading, social media drugs and ai companions, bad code today can and does ruin lives.
See above and the UK Post scandal. Nobody gives a shit. There is no acountability for bad software. Software is not a product from a liability point of view.
> Software engineers, like every other engineer have to be held accountable for code they sign off and ship.
They managers more than them. Software engineers are not responsible for the software lifecycle. Somebody must test the code. Somebody must review the code.
The stakes are now higher with data being so important and the advent of algorithms that affect people directly. From health insurance claims, to automated trading, social media drugs and ai companions, bad code today can and does ruin lives.
Software engineers, like every other engineer have to be held accountable for code they sign off and ship. Their livelihoods should be on the line.