>In the field of software engineering, there are concepts of "clean" and "beautiful" code and languages.

Anyone who has sat in a code review with two "true believer" "software engineers" will know how subjective and poorly justified, yet fanatically defended these positions are - in fact anyone recruiting will know how damaging having a true believer in your team can be (two is a genuine disaster).

Software isn't engineering because we can't measure anything much about it apart from compression, speed and detected defects so far. Speed and compression don't matter for 99% of modern software; we have laptops with 32Gb memory, servers with 1/2 Tb and scores of Ghz cores - the need to write in C has disappeared for almost everyone. Detected defects is problematic because as soon as you start counting them people stop submitting stuff to the code repository until they have removed everything that they can find. That's a problem because they hold onto it for weeks and meanwhile if it had been in the repo everyone would have found twice as much. Also the team stop reporting the stuff they have found and just talk about refactoring; also the team gets less keen on testing..

So what do we measure ? We can't tolerance it; we can't deduce structural properties; we can't deduce lifecycle (recycling, onward development) properties. We don't know why the core engineering decisions got taken because often the core decisions happen at 3:17 on Tuesday and no one even notices at the time, or remembers later. It's not engineering, not close.

> we have laptops with 32Gb memory, servers with 1/2 Tb and scores of Ghz cores - the need to write in C has disappeared for almost everyone.

I don't know in which weird world you live but most of the people I ship software to run 10 years old laptops or 10.6 / 10.7 macs. And even top of the line 2018 macbooks don't have options for more than 16G of RAM.

Not only that, much of software developed today is targeting the cloud where resources are still very expensive.

and those ressources are usually shared between systems!

Increasing memory capacity is not a sound argument that most people no longer need to write C. Most people don't write C because it's easier to abstract away the knobs C provides for faster productivity, not because we have so much hardware we don't need to care. But it sounds like you're saying most people don't need to write in C because they don't need to be efficient, which is silly.

I despise running non-native desktop applications on a workstation with 128GB RAM because the developers often think it doesn't matter how much memory they use if there's a lot of it. That's frustrating and lazy, and it means I'm paying for their time savings with my hardware resources. "Unused memory is wasted memory!" they say, except I would happily use as much memory as possible, but the nth Electron app prevents me from running something else.

C and its derivatives have disappeared from web applications (particularly the frontend), but it's painful to forgo them for desktop and systems applications.

>That's frustrating and lazy

I think it's frustrating (for you) and cheap. For them writing in Python and not C is probably (see my previous comment for my acknowledgement of the lameness of "probably") significantly cheaper - possibly several orders of magnitude cheaper. For you 128GB RAM is pricy, but nothing like the cost of software written "the old way". You are paying for their time savings with your hardware, the alternative is to pay for their time which would be way more expensive.

You definitely have a point wrt. Systems apps; C++ is needed if you want to go close to the metal. I think that Julia is pushing it out of a lot of scientific apps though.

I'm happy to pay for productivity. I'm not happy to pay for inefficiency.

I don't agree with the argument that measurement is the soul of engineering. Even if I did, the argument that we can "only" measure 3 things is itself silly: there is a boundless sea of measurements one can take about everything and anything.

Engineering is the selection of technological solutions to social and business problems, made after some analytical process rooted in mathematical and physical laws, from a set of defensible alternatives, under economic and ethical constraints.

Regardless of what our gatekeeping cousins might wish to believe, software engineering fits this definition.

That many people working on software do not behave like engineers does not change the fact that it exists. That there is rarely any legally-enforced guild structure to regulate the membership of the profession does not change the fact that it exists. Civil engineering existed thousands of years before the word engineering was even invented.

Measurement isn't the soul of engineering, but modelling is - the ability to describe and refine a system mathematically without having to build it physically.

There isn't much of that in application software engineering. Some domains have it, like ML and DSP, but most of the code written for startups is ad hoc carpentry, plumbing, and masonry, not applied physics.

You can do a lot with carpentry, plumbing, and masonry without really knowing what you're doing - up to a certain scale. But when you start building giant things without rigorous non-tribal reasons for your design choices, you're only ever going to build things that fall down and break a lot.

Too harsh? I think it's hard to find anyone today who thinks software is acceptably robust and reliable by default. Some products/apps/pages work just fine if not stressed too hard. But generally security, bugs, and efficiency are all poor compared to comparable challenges in hard engineering disciplines.

Civil engineering is much older than formal modeling. Mechanical engineering is younger than the civil one, but still older than modeling. In fact, when the modeling fad came from the study of the sky into everyday stuff, there was even process engineering happening on the wild.

The only reasonable definition of "engineering" that I know of is on the lines of "the art of building stuff". Writing software fits that definition perfectly well, it's just what people call "software engineering" that doesn't.

Yep this entire thread is a series of arguments over the definition of "engineering." The underlying point seems to be "engineering is serious business and software is not," and the "definition" of engineering is used as a bludgeon to make that point.

This is a topic that really interests me, I'd love to squeeze more info out of you!

Can you elucidate about the sea of measurements that you can extract from software? What are the things that can usefully be measured in the current art?

>made after some analytical process rooted in mathematical and physical laws, from a set of defensible alternatives, under economic and ethical constraints

Ok - how can we do analysis of things that are not measured? In the end all the software design I've ever seen comes down to "this is the best we can do honestly" there could be 100 better ways or this way could be horribly flawed, there's no way to figure this out a-priori.

When I went to college I roomed with a bunch of Aerospace engineers. They spent days and weeks doing structural calculations which were underpinned by physical knowledge of the materials that were in the structures. If the actuator lever for the flap was too heavy or not strong enough when it was made of aluminium then it could be made of titanium instead; but they knew. The sums showed them.

I did CS; at the time we thought that formal modelling would do the same for us.. that is beginning to come back with thing like Lamport's work on TLA+ but I think that the last 30 years have shown us how little we know.

Now yes, people built cathedrals in the middle ages, and bridges. Some of those are around as monuments to their craft. But a lot of them aren't, a lot of them fell on people's heads.

I once watched the software architect at a company I worked for get called out on why he was choosing a certain design decision that was going to impact a critical component of the product, and he justified himself by saying he had a good feeling about it. When pressed further he could only vaguely and emphatically respond “I don’t know, I just think this is the best way”.

So this important decision was based on this dude (excuse me, software architect) shrugging and saying it feels good.

it could mean he's developed a sense in the "right side" of his brain which isn't logic-based. Unfortunately, it's impossible to tell whether someone's taste is bullshit or finely tuned. The best-in-class typically do not follow axiomatic rules either. In this case it's probably the former.

And it's great when someone like Johanna Weber has a brilliant insight about a delta formed wing and it turns into a beautiful outcome like Concorde - if all the calculations are around to un-bullshit it to bits !

Sometimes that’s the best someone with better intuition than communication can produce. (I prefer to err on the side of clear testable communications, but there are weak communicators who are still usually correct)

At least the architect in question was honest. It would have been so easy to spout some generic BS about "industry standard", "principle of parsimony", etc. etc.

True believers are a problem indeed - people who seem to think software development is about applying firmly held beliefs out of context can really screw up things royally.

That being said, by now software engineering has a large set of techniques, with reasonably well understood properties, and we can absolutely reason through whether a technique is useful to solve a particular problem.

For example, polymorphism is a useful technique to invert the locality of implementation details, making it easier to extend software in a distributed way. Its advantage is its drawback, it makes control flow harder to follow and generally the software more difficult to understand precisely due to the distributed extensibility.

Similarly, mocking is a technique to reduce testing "surface", by removing dependency on specific components out of a test. It's a great tool to make tests easier to write, improve test performance, and avoid ambient test dependencies and associated complexity. On the other hand, it reduces test depth by ignoring potentially important interactions, and often leads to overly brittle and hard to maintain tests for little value of tested software.

It's easy to go on. All these techniques aren't silver bullets or axioms to be followed religiously, they are specific solutions to specific problems. They aren't necessarily measurable in an analog fashion, such as the load bearing capability of a steel beam; they are often more binary (quite fitting, given the domain). But you can judge whether they are useful to solve a specific problem.

That is, I don't follow the "it's all just opinion" kind of software engineering defeatism. Yes, there's a ton of extremely questionable evangelism, often spread with huge pomp and circumstance, by people with limited insight. But that's really just that, limited insight.

Others have replied to your point on performance being irrelevant - that's another area of software engineering that's hugely important, and not just in the Google kind of setting.

Dr. Hamilton (who coined the term "software engineering") developed a system of software construction she called "Higher Order Software" distilled from her experience writing the code for the Apollo 11 mission.[0]

Unfortunately, her insights have been largely ignored. I first heard about it twenty years ago in a book called "System Design from Provably Correct Constructs" (which, oddly enough, doesn't mention Hamilton outside of a few references in the back.)

In modern terms, programs in HOS are developed by the direct elaboration of the Abstract Syntax Tree (like Lisp code) using only a limited set of operations which each preserve the semantic correctness of the tree. In this way, you are prevented from building incorrect software.[1]

(You might build a barn instead of a pool, but it will be a fine barn.[2])

In addition to the availability of a defect-free development methodology, Category Theory provides a solid mathematical foundation for determining the most-highly-factored form(s) of a given code-base. Meaning, if you have a "categorical" programming language you can more-or-less automatically extract the most general form of your program, which can then be used "over" different categories to get different concrete programs from one piece of code, with confidence (mathematical confidence) that they are valid, correct programs. There's now a great explanation by Conal Elliott about this called "Compiling to Categories"[3].

In any event, I would say that Category Theory gives us a language-agnostic way to rigorously define the "cleanliness" or "beauty" of software.

The tools are there. We have to notice them and pick them up.

[0] https://en.wikipedia.org/wiki/Margaret_Hamilton_%28scientist...

[1] You're not editing text files and then using other tools to (try to) ensure they're not broken too badly. No linters, no formatters, no syntax errors, no type conflicts, no off-by-one errors, etc... Hamilton identified all the sources of error in program construction and designed a system that eliminated them.

[2] "'Tis a fine barn but 'tis no pool, English."

[3] "Compiling to Categories" Conal Elliott February 2017 http://conal.net/papers/compiling-to-categories/

I'm sorry I've spent half an hour with [3] and it's pretty challenging. Is there something with training wheels?

I don't know. It's kind of newfangled. It's "in the air" though, I think. Here's what I got (it helps to know a little Category Theory, try Bartosz Milewski's "Category Theory for Programmers"[0] if you don't already.)

In "Compiling to Categories" what he’s doing is translating lambda forms into a kind of “point-free” style and then showing how to instantiate that code over different categories to get several different kinds of program out of the same code.

I've been working with a stack-based "concatinative" language called Joy which I think makes this stuff a lot clearer. Joy code is already in "point-free" style (no vars, no lambdas) and it includes functions that do the same job as but are more elegant than the triangle operators and others in the "squiggol" tradition. ( https://wiki.haskell.org/Pointfree )

There's a little bit in "Mathematical foundations of Joy"[1] and "The Algebra of Joy"[2] by Manfred von Thun.

Here's a piece of Joy code (it's part of an ordered binary tree library):

    pop swap roll< rest rest cons cons

Here's a trace of its evaluation (with a suitable value already on the stack) in the category of values resulting in a computation of an answer. The dot is the "interpreter head", the current point of evaluation:

    [4 5 ...] 3 2 1 . pop swap roll< rest rest cons cons
    [4 5 ...] 3 2 . swap roll< rest rest cons cons
    [4 5 ...] 2 3 . roll< rest rest cons cons
    2 3 [4 5 ...] . rest rest cons cons
    2 3 [5 ...] . rest cons cons
    2 3 [...] . cons cons
    2 [3 ...] . cons
    [2 3 ...] . 

And here's the same code evaluated in a category of composition of stack effects resulting in a description of the stack effect of the expression:

    (--) ∘ pop swap roll< rest rest cons cons
    (a1 --) ∘ swap roll< rest rest cons cons
    (a3 a2 a1 -- a2 a3) ∘ roll< rest rest cons cons
    (a4 a3 a2 a1 -- a2 a3 a4) ∘ rest rest cons cons
    ([a4 ...1] a3 a2 a1 -- a2 a3 [...1]) ∘ rest cons cons
    ([a4 a5 ...1] a3 a2 a1 -- a2 a3 [...1]) ∘ cons cons
    ([a4 a5 ...1] a3 a2 a1 -- a2 [a3 ...1]) ∘ cons
    ([a4 a5 ...1] a3 a2 a1 -- [a2 a3 ...1]) ∘

If you compare the input and output of the first one you'll see that it matches the input and output stacks in the Forth-style stack effect comment computed by the second one.

To repeat, that's the same code evaluated in two different domains, er, Categories, to get two different correct computations.

If I had implementations for them I could evaluate the expression in/over a category and get as output: a dataflow diagram of the program, or a hardware description of a circuit for the program, which are examples from Elliott's talk+paper.

[0] https://bartoszmilewski.com/2014/10/28/category-theory-for-p...

[1] http://www.kevinalbrecht.com/code/joy-mirror/j02maf.html

[2] http://www.kevinalbrecht.com/code/joy-mirror/j04alg.html

I was with you till the 32Gb memory part. In what part of the world do users have devices with 32Gb RAM(barring gamers)

I did stray into polemic, but to be fair any serious user, including gamers should upgun their memory. It's cheap and effective. Also one of the complaints about mac book design has been that there is no 32/>32 memory capability!

This article [2015] is of relevance in this context. It refers to the work of Richard Dawid who's apparently laying out an expanded view of science beyond the commonly accepted Popperian perspective centered on falsifiability. In his 2013 book "String Theory and the Scientific Method" he apparently identifies "three kinds of “non-empirical” evidence that Dawid says can help build trust in scientific theories absent empirical data." Of course, this is not an accepted view yet, but it can lend some balance to the conundrum resurfacing today.

[2015] https://www.quantamagazine.org/physicists-and-philosophers-d...

For me, "beauty" is an inverse function of entropy or disorder. The fewer the arbitrary constructs in a system (or theory), the more "beautiful" (or more accurately, elegant) it is. After all, the laws of physics are about finding recurring patterns in a mass of random observations. So I think it's reasonable to prefer theories that can explain a large number of random observations with a fewer number of laws. However, I suppose the argument of the book is that when we try to make our theories more simpler than reality actually is, we have a problem?

Edit: to try to answer your question, the goal of elegance/beauty in code is entirely about comprehensibility and not utility, is it not?

"As simple as possible, but not simpler." (often credited to Einstein)

See also https://en.wikipedia.org/wiki/Occam%27s_razor

That said, the whole field of science journalism and popular science book writing lives from violating these rules, making things much simpler than they actually are. But then again, the average person is a) not in charge of the physics experiments and b) tries to use these as a reference in a scientific paper is up for a surprise.

Experimental physics is not “a mass of random observations.”

I wasn't referring the experimental physics. I was referring to how we perceive the wold in the absence of physical laws.

Beauty is in the eyes of the developer.