Yet there are quantitative metrics for engineers. Is your Ruby on Rails application really more complex than the software flying an airliner?

There are metrics for problems which are studied in-depth before engineers are let loose to solve it. That's not true of most software projects.

Who do you think does the studying? Engineers...

You're agreeing with the parent. There are metrics to measure solving well-known problems, but no metrics for researching problems to make them well-known.

But the first person or team who realized that you could put a computer in an aircraft weren't solving a known problem - however they were using engineering methods, which is why it's very rare for avionics to (literally!) crash.

Here's my theory: we do know how to build very high quality software in a repeatable way. However very few organizations are willing to pay what it costs to do that, and that is why most software is so bad. Admittedly a lot of people writing code these days are "cowboys" (they don't care and neither do their clients, they just want speed) but that doesn't mean that the body of knowledge doesn't exist.

I don't know why this comment was voted down, but I think he could be right -- given the correct resources (time mainly), and assuming the right talent level, I think we could consistently build rock-solid software. The constraint seems to be a (logical and necessary) trade off between total cost and total utility.

As we all know, something that works 95% of the time at 5% the price is very often better than a "perfect" solution. With critical systems like plane navigation, the tradeoff doesn't make sense, so you pay full price for near-perfection... your niche e-commerce site? 95% is sufficient.

Plane navigation is in someways easier to e-commerce sites as well. The environment that the plane is in, isn't actively malicious. You can collect accurate statistics on weather, where as the attacks and usage (with bots/scraping etc) consistently change.

Not at all man. The physical environment can be much more malicious than you seem to think.

I've done some software for buses, it's toy in comparison (with planes) but you get issues with noise, power supply, vibration, temperature, light (UV killing a touch screen) and probably lots more i've never encountered. The physical world can be every bit as unpredictable as the virtual one.

The major difference is that environmental agents are generally deterministic or stochastic, i.e. obey the law of physics. On the other hand, realms with intelligent agents, such as humans, are inherently unpredictable due to their free will (which is unpredictable by definition).

You also have the free will of humans messing with the system (in my case, fraud prevention), also consider animals interacting the machinary while someone is not watching. I remember hearing a story about a huntsman spider regularily tripping a sensor for a program that measured the weight of lorries.

The only consistent thing you can state is that all these "real world" issues are to do with hardware whereas yours are more softwarey. However the consideration of such issues is necessary across the board.

I don't know about you but if I was writing a program to fly a plane I'd want to make damn sure I took into account an intelligent malicious agent trying to crash my plane.

E-commerce has been around for >15 years now, and there how many tens or hundreds of thousands of e-commerce sites? It's a very well known problem by now.

regarding problems and engineering, lets see how brooks puts it:

"Essential complexity is caused by the problem to be solved, and nothing can remove it"

so the kind of complexity with have to tackle comes from the area of application we are in, not the programming task itself. its not about writing pretty, bugfree java code. let's assume your java is 100% perfect. there still remains the essential complexity of the problem.

now if the problem is in any way related to humans (a lot of software has to be 'a solution' for humans) then there is no way to proof a solution to be The Right One. hence B/B testing. this is at some stages the way to got.

show something & get feedback from the user, who knows more about the problem (himself) than you. this of course doesn't mean it's all blind trial&error - well in a way it is, but humans get better at that over time.

disclaimer: if you program the fuel-control for a rocket that is 30 years old, you can of course calculate the correct solution and resolve all essential complexity to mathematical formulas.

interestingly this was not the case for the first rocket-programmers, since the rockets kept changing every few month :)

Look at slide 20 again: every 25% increased problem complexity results in 100% increased solution complexity.

It's more open-ended because you can mostly make your own rules, unlike in the physical world where you have lots of predefined constraints. The resulting solution space being much bigger, you have to use your taste to narrow down to the better solutions, so of course it's more subjective.

What's your point? Does airline software have a good metric?

The main difference is that other engineering disciplines don't have to deal with the halting problem.

I don't think it's a matter of complexity so much as a matter of difficulty defining the problem.

I agree, But what if we looked at a job where productivity should be much easier to define, like a roofer.

An obvious performance metric would be "Time it takes to install a roof that will pass a quality inspection". That metric is pretty useless alone though. Can he fix a nail gun that breaks? How safe is he? Can he work as fast in cold weather? are just a few factors.

Say you could get metrics for ALL productivity factors, you still haven't taken into account personality issues that cannot be quantified.

And this is for a single person working on a measurable task with only a few tools. What chance is there for metrics to be accurate about a task that could require any number of tools, multiple correct answers, and collaboration?

"Does it work" isn't a quantitative metric, it's a qualitative one.

For sure. Before 'hit parades' music could sit around in a desk drawer for decades (Schubert) before being heard. Because innovation drove away audiences wanting to hear the familiar. The people who made sure that Schubert's stuff lived was other composers ... not audiences.

Software's the same ... the people who cheer great code (art, science) are people who understand it well enough to recognize great code. The opinions of peers is an important part of achievement metrics. And that artful kind of code pretty much counts for nothing by any standard 'industry' measures. But it's influential.

I agree this is a key point. Software bug rates are not comparable to scurvy incidence rates, and developer productivity is not similar to the number of bricks a laborer can move in an hour.