You want a 100% bug-free web browser? OK, so let's define precisely and formally what the browser is supposed to do. Then, you'll sign a very expensive contract, and in 6 months to 1 year, I'll deliver the software, with a formal proof it works as expected. Of course, the requirements won't change during the development process, and any update after delivery will be costly and will take time.
Now, who would want to pay thousands of dollars for a web browser that will be delivered in a year and that will be obsolete as soon as a new fancy web tech will be used everywhere else? I can't really blame the customer/client either.
But the thing is, "agile" requirements rely on craftmanship, not on engineering. You can only engineer durable things. Buildings would crash constantly, too, if they had to be updated each other week, ASAP, and while keeping the costs as low as possible.
If we're going to build something non-trivial that is correct, we first need to specify what a "correct" result would be, in some rigorous, comprehensive, unambiguous form. That in itself is already beyond the vast majority of software development projects, though not necessarily outliers like very high reliability systems.
That is partly because the cost of doing so is prohibitive for most projects. This is a common argument against more widespread use of formal methods.
However, it's also partly because for most projects the desired real world outcomes simply don't have some convenient formalisation. The potential for requirements changing along the way is just one reason for that, though of course it's a particularly common and powerful one. There's also the practical reality that a lot of the time when we build software, we're not exactly sure what we want it to do. What should the correct hyphenation algorithm for a word processor be? How smart do we want the aliens in our game to be when our heroes attempt a flanking manoeuvre? If you're a self-driving car on a road with a 30 legal limit but most drivers around you are doing 40, how fast should you go and why? Once we get into questions of subjective preferences and/or ethical choices, there often isn't one right answer (or sometimes any right answer), so how do we decide what constitutes correct behaviour from the software?
It should do a bunch of really creative things then further new fancy web tech shouldn't exist. Something like: Distribute documents signed by their author in a truly robust way. It should have A review and certification system and It should help monetize publishing in a truly transparent way.
Write it in hardware design language and run it in an emulator. :-)
If it really took a year to make a bug-free browser, we would have 100 of them instead of 0.
I was told that some one had said well this protocol is all very well but we need to build our own slightly different improved version.
Luckily this was stomped on from a great Hight by senior people at BT Labs
But who would claim that seriously? That would be obvious hyperbole. The claim is rather that certain teams of engineers have created software/hardware combinations that, for all we know and for all practical purposes, did not have a bug and have not failed due to software error.
The development of high integrity systems is costly, though, so the discussion is a bit moot. Sure it would be possible to develop an Android app that - within the limitations of those devices and the buggy operating systems - would not fail due to a bug in its own program. It's just really expensive, particularly if the software is also formally verified. Generally speaking, it doesn't even make sense to consider "high integrity software" without the accompanying hardware. You can only satisfy real-time constraints for specific hardware anyway, and the certification and evaluation should be for software+hardware.
All humans are capable of unkowingly making mistakes which can have dire consequences. We've yet to see an approach to software development that will, without a shadow of a doubt, erradicate the possibility of bugs.
Project managers constantly change their mind about requirements and then wonder why we're shipping late, although I suspect some of this is not considering failure/bugs when planning. Even when it comes to breaking down and estimating work, if the estimates software engineers produce don't align with the high level schedule they're automatically wrong or people start asking about what we can trim down (which somehow is always testing).
Perhaps it's idealism vs reality, but I find it a bit bizarre.
Devs are expected to have a ticket to track everything they do, so should the product managers. There should also be tickets for meetings so I can track how much time was subtracted from development by useless meetings.
In my experience, this could not be further from the truth. There is simply no awareness that change remains expensive it's just that now the process allows it without any fanfare whereas before a change was a big deal.
Not saying that change requests are the way to go but certainly an understanding that change is not free would go a long way
Maybe there’s not as big a difference as people think, though?
It occurs to me that is actually possible to change the design of a bridge late in the day. The extra dampers to fix resonance problems on London’s Millennium Bridge are a nice example.
(I don’t actually know which it is; maybe it’s both)
It's extremely dangerous, and software is quite comparable to a bridge. Imagine starting at both sides of the river and the bridge not meeting in the middle.
Changin requirements / features is very simular. And while on paper it sound like no problem, thats when you need to push back against your PO/Whoever is pushing the change.
I'm not sure what the point is, but it somehow fits into yours...
We can pretend we're building a bridge, but there could be another company, that is more comfortable with changing requirements and may deliver their product later, but more suited to a market.
Ultimately all software becomes very inflexible if you cannot get the source code or especially flash firmware.
Software looks as if it should be more flexible than concrete.
But it really isn't.
You are, of course, absolutely correct. Context is key here: you treat software engineering like "real" engineering when it's appropriate to do so (space flight, aircraft control systems, nuclear power plant monitoring and control, medical devices, and so forth).
However, I think for many organisations - and for much of the time - the engineering aspect is often heavily overplayed, and actually you're trading that off against two key aspects of software that can be used to build a real competitive advantage. Those being (i) speed of implementation, and (ii) malleability. Building software can get you to a solution very quickly. You can also iterate or modify that solution very quickly.
Clearly there are limits to this but for many companies, and most startups, getting something working quickly, and iterating on it quickly is far more important than any engineering merit. Same goes for larger companies introducing a new product: don't overegg the "engineering" because it will slow you down and time may be the only sustainable competitive advantage you have.
There's a time to be an engineer, and there's a time to be a cowboy hacker (and everything in between). The real trick is in knowing where on the spectrum to execute in the context of the customer base you're serving and the software you're building.
Nobody ever asked me about what it takes to ensure it is correct. Nobody asked me to work with them to produce a detailed specification that exhaustively considers all cases that must be accounted for, nobody asked for a formal proof, nobody asked me to run a model checker, not even do something like design by contract. And when I look at job postings, all this stuff is completely absent. I was excited when I found one company that recognized Ada and Spark, and got me into an interview for mentioning them.
Just sprinkle some assert and make a few tests. If shit stick to ceiling, it's cooked. That's how the industry works.
We aren't even trying. It's pretty funny that everyone's making claims about what's possible when nobody is trying.
Yep, I bet there are niches, like in aerospace, where some try, and I'm sure some of them also succeed at correctness. We only hear about the failures.
They don't even know about correctness without testing, they tie correctness and testing together and think it's the same thing.
>There are industries where it is important and where it is done routinely, but evidently not in the places most HN commenters work.
Can you name these industries?
The notion that most(*) any non-trivial program can be formally specified is a myth that largely exists in educational institutions or research institutions disconnected from the realities of production software engineering, and has no basis in real life. Yes, there are a handful of exceptions for extremely constrained control problems in avionics, and even those turn out to have errors in the specification.
What we like to call "software engineering" isn't actually engineering, and more like a craft than anything. Actual engineering in software is possible.
I take issue with calling our field software engineering. It's a buzzword.
Having now left the field, working on most other software systems and teams tends to basically be "code-til-it-kinda-works" (despite the paeans to "Agile"). It's been amazing how just applying a little bit of the rigor I developed from earlier on in my career from working on embedded flight systems pays incredible dividends later on.
That said, one should have no illusions that NASA is this wonderful hub of innovation and a model for organizations to emulate. Not at all. Definitely not. But there is something to learn from a track record of consistent success flying computers deep in space.
...Got something to share with us?
;)
That doesn't mean incorrectness isn't there. That means the circumstances necessary for any incorrectness to manifest in a measurable way didn't happen.
We know that now, but it's an example of a rule that was not written in blood. Many are. Someone had an intuition and nobody died because of it.
No software is trivial if the author wants to broaden his argument to the human species.
Basically, if you constrain your software to work for your specific use case, in your specific system, it's "100x" easier.
Also you don't know how many bugs, known or unknown exist in software on space probes.
For most applications, is it worth spending several times more time and resources in order to make it "correct"? I don't think so.
Does it correctly handle every possible sequence of inputs? For the vast majority of software in use today, the answer is "no"; The follow up question is "does it matter?" and (luckily or unluckily) the answer for the vast majority of software in the vast majority of use cases, is "no" as well.
But occasionally it does, e.g. https://en.wikipedia.org/wiki/Therac-25 is a famous poster child, and I quote:
>>> The failure occurred only when a particular nonstandard sequence of keystrokes was entered on the VT-100 terminal which controlled the PDP-11 computer: an "X" to (erroneously) select 25 MeV photon mode followed by "cursor up", "E" to (correctly) select 25 MeV Electron mode, then "Enter", all within eight seconds
Software was not obviously correct or incorrect; In fact, it had been acceptable for an earlier model (which had some hardware protections missing from the newer one). Reaching the incorrect state required the race described above to trigger, which did, in fact, happen in practice a handful of times.
You can work very hard to formally prove your implementation, only to find out that the compiler had a bug and makes your software bad. Or the CPU does; Or all the designs are fine, but there's an bit flip due to electro-migration or cosmic rays. Many people consider this "force majeur" - "an act of god" one cannot anticipate, but cosmic rays are in fact an expected -- and hard to avoid -- input to many systems.
You are in control of a logical model which you can, with extra work, do (provably) correctly. But that IS, from experiencce, 10x to 100x more expensive, and unless you go for 10x-100x more expensive hardware, reduces and moves the sloppiness factor around, but does not eliminate it.
I think in that light, most modern software can be built to be robust and easily maintainable rather than correct. Like the difference between building a bridge, where you understand everything in the system, versus building a vehicle engine, where you have multiple dependencies but you still need it to not catastrophically fail should any of them stop being correct (eg: fuel pump fails, engine management detects lower fuel pressure and cuts ignition, saving the engine).
I don't know a single piece of software on my computer right now that doesn't crash, act weird, hang or do something else to annoy me on a daily basis.
Now yes, each infraction might be just a minor niggle, but god it adds up!
By the end of the day, all the bugs that on their own shouldn't be too bad, leave me frustrated as hell and wanting to say fuck everyone and just go live in the woods.
It's especially worse when I know this shit doesn't need to happen.
It's all about tradeoffs. Everybody wants cheap software that works perfectly, but reality dictates that can't happen. Most of the time, cheap/free with minor annoyances is better than perfect but high-cost.
We should be worried about poor development practices in the automotive industry specifically. Looks like many manufacturers approach the tasks the same way any other hardware manufacturer does, i.e. software is secondary in their minds. Software for a car is kind of the same as software for your next "smart fridge". Among other things, it can be outsourced for example.
Whereas, as Bruce Schneier put it, a modern car is a computer on wheels, not the other way around. A computer on wheels is a lot more dangerous (and there are security implications too, it's what Schneier meant actually).
The Mars Rover has specific requirements, is this more trivial than a startups new web-app?
Rephrasing it as something like it's incredibly difficult to write a large piece of software with no bugs makes more sense. And I think for most organisations it may as well be impossible.
Software is becoming more and more depended on for life and death use cases every day.
I feel that migration is often ignored or discounted. For example, Facebook in the early years was considered mostly harmless, but now has migrated to being exploited by state actors to brainwash populations into hating each other, interfering in elections, or at worst performing genocide on a minority group. We need to stop assuming that just because a software application is harmless now, that it will stay that way, and we need to adjust its "correctness" accordingly as it migrates along the harmless <> life-and-death scale.
In my experience, nearly all delays in software shipments have been only in the eyes of idiot managers: developers and smart managers recognize the constraints being constantly added to a project and know in advance that the result cannot be what was promised in a totally different scenario.
Especially with the plethora of stories about fixing bugs in probes mid-mission, crashes due to poorly sanitized inputs, and byzantine failure conditions.
I mean, if correct = 100% uptime, 100% effectiveness, etc., then it's not so much of a software problem, but a hardware one.
BTW, "is it software or hardware?" is exactly what the boss asks first when a customer has a problem.
When author (or most people, I believe) mentions software, it seems it's just apps (like food apps mentioned elsewhere) or web apps, that is just a fraction.
Even if the software was 100% perfect, such a machine is expected to crash once or twice every year or so under full load just from the failure rate of the chips (cosmic radiation / quantum tunnelling effects). Especially RAM seems to have become fairly error-prone nowadays.
My Viaplay app stops streaming and tells me I'm not connected to the internet several times a day (when I'm obviously still connected to the internet.)
Several times a week my phone's Netflix state doesn't match my TV's state, so I'm either not presented with the controls or when I press them they do nothing.
Cosmic bit-flips are rare enough to be newsworthy. https://www.johndcook.com/blog/2019/05/20/cosmic-rays-flippi...
If the faults were with the hardware, I'd expect the errors to be spread around the software that I use. But Firefox and Gmail always work for me (even if the JavaScript I'm served makes the machine unresponsive), whereas Netflix, Viaplay and HBO constantly misbehave.
This can be done and has been done but is not well known among the javascript boot campers that populate HN.
I don't think it's more complicated than that.
I'd argue that it changed once smaller businesses could afford (and have room for) their own computer. My best guess would be somewhere around the PDP-8 era (late 1960s-early 1970s).
Once things started to move away from large-scale computer rooms and consoles of blinking lights, towards a more "hands-on" and interactive approach, where the programmer didn't have to wait between "batches" of runs to see what their code did (correct or not), that is when (from a software development point of view) so-called "engineering" went out the window.
What would be the quintessential differences here? We plan, we test, we revise. I'd argue we're all doing the same thing, but if you're working on a food ordering app you can afford not to be as stringent with the testing and quality, versus software that takes people into outer space.
It's just a matter of stronger concerns. Things matter more in a field where you can kill people if you have a bug, so everyone strives to do a better job, top to bottom.
Most food order apps seem to be more like, eyeball it, slap something together, then it works until it doesn't, at which point you figure out a fix. That works for a while until something else breaks, so you patch a fix into that. Ad infinitum.
Agreed. Sadly I'm much more interested in engineering quality than revenue..
And NASA software is not that complex if you think about it; at least it has fixed requirements; once they ship it, it's done. No more changes. This is different from some complex software which require constant changes and adaptation to changing requirements; in this case the code has to be designed to adapt easily; this is difficult to achieve and most developers/companies fail at that or in the case of companies, they only just barely succeed through the development of extremely expensive solutions.
Big companies like Google don't write efficient software; they use their capital to hire as many expensive engineers as possible to brute force the requirements until it works. That's why they keep rewriting projects from scratch every few years over and over; they don't know how to write adaptable software. Their software is somewhat complex, but still simple enough and they have enough capital so that brute forcing solutions are still feasible (for a high cost).
The same cannot be said about more complex tech like blockchain or large-scale machine learning systems; you can't brute force your way to a solution and full rewrites will actually set you back.
Nope, they still make changes. This is a case study for a company that helped debug and fix a low-level C bug on the Mars Curiosity Rover as it was traveling to Mars. https://semmle.com/case-studies/semmle-nasa-landing-curiosit...
I work with high-assurance software in the blockchain space and only recently read a paragraph from some old NASA software quality textbook and thought damn.. this is awesome. It's anything but trivial!
Bullshit. Humans have been sending computers into space for decades, and while yes, some have had programming problems, most worked correctly, for a very specific definition of correctly.
Thing is, NASA (and Russian and Chinese and Indian and ...) space engineers have approached their challenges from the point of view of actual engineering, while modern "software engineering" is all about craftsmanship, and just a sprinkle of actual engineering to make ourselves feel good.