If you ask developers to write efficient code and give them the means to do it, they will write efficient code.

But unless they work in embedded software, this is rarely a priority. In fact, this tends to be actively discouraged : in the good shops, reliability, robustness and security is the goal, in bad shops, that's bullet-point features and meeting unrealistic deadlines.

To get an idea of how optimization is poorly regarded, look no further than the infamous Heartbleed exploit of OpenSSL. One of the contributing factor for this bug was that the team used a custom memory allocator for performance reasons. After the bug was discovered, the team was actually blamed for it more than anything else that lead to the bug. And FYI, it was a real performance improvement, at least on some platforms.

You seem to be under the impression that writing efficient code is at odds with writing reliable or robust code. Efficient does not mean unsafe. Efficient code does not have to shoot-from-the-hip. Efficiency means that the task is completed with the minimum wasted effort. To assume that you cannot have relieable code that is also efficient is absurd.

If reliability is the top priority for your project, it just means you factor that in to your optimizations. It doesn't mean you just ignore efficiency and write any old shit, as seems to be the custom today.

Your OpenSSL example seems to suggest the Heartbleed exploit was a result of developers daring to write performant code, and not that they simply wrote buggy or insecure code, performant or not.

I feel the problem here is what I imagine as this formula:

  reliability * efficiency = developer skill

I.e. for a given level of reliability, you need more skilled[0] developers to be able to write efficient code. And I'm not thinking about premature optimizations here - just that the code an experienced developer will write the first time around will be efficient enough.

So in short, if you aim for certain level of reliability and employ cheap/unskilled developers, efficiency will suffer. And as long as companies don't really care about performance, developers will not gain the necessary experience to write efficient code with just-in-time optimizations.

--

[0] - for "skilled" meaning probably mostly some knowledge and experience in writing fast code; it's probably not the experience you'll get when all coding you've ever done is for the web.

I think he's referring to the pressures to produce working solutions. And there's a whole laundry list of attributes people attach to that process. And everyone has the opinion that they're ideals represent the best/most important subset of all the possibilities and/or that you should just meet them all or go work at McDonalds.

You can certainly write efficient code, that's reliable and robust and secure given the time and resources to do so. It usually comes down to money, and how no matter how efficient or high minded you are, crafting the optimal solution takes more time and resources. IE more money. If you can sell users and clients on the extra costs, great. Otherwise you need to figure out what you can do, with the time and resources you have, that will sufficiently satisfy your customers.

And yeah, the problem is with the gobs of resources available on modern computers writing efficient code matters less because most users aren't going to care if their music player is using 1% of the CPU or 10% of the CPU.

Of course I say that with the annoying exception I've experienced where the custom MP3 player in GTA V requires a relatively huge amount of resources and will easily reduce performance by 10-20FPS even on an overclocked i5.

>You seem to be under the impression that writing efficient code is at odds with writing reliable or robust code.

For a given amount of resources invested it is. From a business perspective this includes not just the time a developer spends on a given piece of code, but how good a developer one is willing to pay to work on the code. Perhaps those making the business decisions overestimate the cost of writing more efficient code and underestimate the costs of not writing more efficient code, but they there is a cost that will be taken into account.

>To assume that you cannot have relieable code that is also efficient is absurd.

Of course you can, but often it takes more time (or more expensive developers) than inefficient code, and the people paying the bill don't necessarily benefit from the efficiency.

I want to believe this. Evidence shows that people tend to grow their problem so that they are attempting to solve the problem of writing software with every other developer. Hence the rise of frameworks.

Sometimes, people get a knock out and do a good job. Bootstrap was seen that way for a while. (I've not heard much of it lately, so assuming it has waned in popularity?) By and large though, we all have different requirements for development. And none of them are shared by the customers/users of our software.

There's also this weird collectivist thing going on with software development, where not using frameworks is looked down upon. Or rather, frameworks are considered a best practice without taking project requirements into consideration. Sometimes, said developers then blog about how they used a nailgun to swat a fly, and the hype cycle continues.

Example: everybody wants their web pages to be smaller but nobody wants to part with massive JS frameworks for sites that everyone are using, even if their site is mostly text. Instead, endless amounts of blogging and engineering time are dumped into things like minification, compiler tech, and other esoterica to solve a problem created by developers. This is incidental complexity, and it lays waste to software systems.

In short, the zeitgeist selects what it wants to hear.

In my observation, this puts the blame in the wrong place. It's simply that most software development is feature-driven, and efficiency is rarely held to be a feature by the business that's paying for them. That is, until it becomes a problem, at which point there are enough low-hanging fruit left by the constant scramble for user-facing shiny that making it not a problem again is relatively straightforward.

I didn't buy that excuse at all. Devs have a habitual problem blaming others for their poor code.

Often better code takes only marginally longer or even less time since you aren't having to deal with all the extra complexity that tends to be a part of "modern" development.

The complexity is of our own making. We spend too much time on this extra cool and extensible way to use json and xml in case we even want to run on a quantum computer in in 50 years or avoid a simple recompilation when we need to change a constant that will likely never need to be changed when we should have been spending that time on more important things.

You're conflating "better" with "more efficient to execute." That's only one possible interpretation of "better", and it's usually not the most important one.

but it very often is. If you're a web developer, for example, little matters more to your users than performance. This has been demonstrated an almost infinite number of times by now, but surprisingly few industry professionals seem to pay any attention to it

Yes, and we can point that out to our business owners until we're blue in the face, but until they see the benefit they won't pay us to do that work. It can be surprisingly difficult to get this point across.

Just because something matters objectively, doesn't mean it matters to the client/business (at the moment). And unless you as the developer are in the position to dictate that, sometimes you're at the mercy of non-technical requirements or demands. You can be right about everything, but unless you're the boss, at least sometimes you can only do what you're told.

Little matters more to your users than performance, specifically, only features and security matter more than performance :)

And that the software is available (ie actually shipped). And to use it they need to know it exists and what it's for, which usually involves some marketing.

I think it unfortunately takes experience to write efficient code the first time around. Experience people are not gaining, since there is little to no focus on performance outside embedded and gamedev circles.

Well for some reason it's those people [1] who have the money so they call the shots.

Do you know of organizations where an internal drive for excellence is cultivated? Where technical people communicate directly across hierarchy levels and horizontally? Maybe even outside the company?

[1]: People who are relatively close to the customer and not the implementation. Those people don't tend to understand how efficient some things are because they can't estimate how long something should take from lower principles which they have a benchmark for. This makes them much happier software users.

"Sometimes the excuse is that cpu and memory are cheap as if resources aren't under constant pressure from the hundreds of processes that all pretend they are the only one on the machine."

Does anyone know if there's a generalized name for this problem? It's the same one where your teacher gives you "only two hours of homework tonight" but forgets you have four other classes all doing the same thing.

Maybe 'Tragedy of the Commons'? [0]

[0] https://en.wikipedia.org/wiki/Tragedy_of_the_commons

Tragedy of the Commons, as mentioned by others, sounds most adequate. In general this is one of the whole class of coordination problems - where actors reach globally suboptimal outcomes by failing to coordinate with one another.

Tragedy of the Commons seems to imply some harm is happening to the actors themselves; the coordination problem with computing resources here is a little different, because most of the time, application vendors don't feel any impact of the problem they contribute to, which makes them even less incentivized to coordinate.

It's not exactly what you're looking for, but I've always been fond of Raymond Chen's "What if two programs did this?" thought experiment:

https://blogs.msdn.microsoft.com/oldnewthing/20050607-00/?p=...

Parkinson's Law

"Originally, Parkinson's law is the adage that "work expands so as to fill the time available for its completion", and the title of a book which made it well-known. However, in current understanding, Parkinson's law is a reference to the self-satisfying uncontrolled growth of the bureaucratic apparatus in an organization."

Hmm, I can see how they're related, but it doesn't seem to be quite what I'm talking about. I'm more referring to the overuse of resources by individual elements assuming or acting as if they're the only thing using it, like in the above examples of a teacher acting like their class is your only one, or a thread consuming resources like no others are active.

Parkinson's law more seems to be the result of the effect that I'm talking about (among others, like procrastination).

Tragedy of the commons?

willful ignorance?

"to throw money at it" is a general expression for the solution to overcommitment, schedule conflicts and resource exhaustion.

I'd call it Assumption of Isolation.

> premature optimization is often used to shut down discussion of how their code can be improved

and

> The field is too in love with horribly inefficient frameworks

Dude, I can't emphasize this enough. One of the best things I read in past week.

And so where can a self-taught programmer self-teach himself about this stuff?

I'm probably guiltier of this than I realize, especially since I loooooove me some abstraction.

If you know C at the level of K&R, a good start would be learning from Bryant and O'Hallaron's book: http://csapp.cs.cmu.edu/

From there, you would be well served by working through Hennessy and Patterson's Computer Architecture and Computer Organization and Design.

In parallel, you could work through a book on Operating Systems, maybe even playing with the code for a toy system (e.g., http://pages.cs.wisc.edu/~remzi/OSTEP/ or https://pdos.csail.mit.edu/6.828/2016/xv6.html).

An alternative or perhaps parallel approach would be to play the assembly games put out by Zachtronics.

A simple thing is to figure out if you can restrict the runtime environment during development (available memory, clock speed, network speed, etc), and develop to that. The nice side effect is there's more of your computer available for IDE's and browser tabs.

Secondly, abstraction doesn't necessarily track with efficiency/inefficiency.

Efficient programming is all about knowing how to use CPU features and memory to your advantage. If you don't already know C, I suggest "Learn C the Hard Way"[0]. After that, here's two great places to start: "What Every Programmer Should Know About Memory"[1] and "What's new in CPUs since the 80s and how does it affect programmers?"[2]

[0]: https://learncodethehardway.org/c/

[1]: https://people.freebsd.org/~lstewart/articles/cpumemory.pdf

[2]: https://danluu.com/new-cpu-features/

Go learn yourself some embedded, if you want to deal with extreme resource constraints, or some gamedev, if you want to know how to write performant code with enough tasty high-level architecture decisions to satisfy your abstraction needs.

"Arduino" and other small-system programming. Or get an emulator and try to write an Atari 2600 game: you have 128 bytes of RAM, don't use it all at once.

I should have been clearer: I'm mostly interested in books and articles. I like to approach things theoretically before approaching them practically.

(Nonetheless, thanks for your suggestion!)

Try "Racing the Beam"; a neat book describing the programming challenges with writing software for the Atari 2600.

https://www.amazon.com/Racing-Beam-Computer-Platform-Studies...

Though, instead of reading, you really should just get over your complex and try writing some efficient code.

>get over your complex

It's not a complex. I'm always very surprised by this attitude. Theory serves practice.

Case and point: the OP mentions that an understanding of cache architecture enables you to reason about which data-structures exhibit good cache locality. Discovering that hashtables have poor cache locality by trial-and-error seems like a waste of time compared to gaining theoretical insights.

I'll second "Racing the Beam", although it's a history rather than a theoretical approach.

I think you may have hit a linguistic snag here; there's lots of "theory" in the classical CS sense dealing with performance from an O(n) point of view, such as Knuth's work. But for producing fast results on actual hardware you end up having to take into account lots of ugly details of the platform. Learning about cache behaviour doesn't really fit into CS so it's not the first thing people think of when you ask for theory.

"Knowledge transmitted by practitioners through written and oral culture outside of the academy": what's the word for this?

Part of the problem is that some of this stuff has a pretty short shelf life. There's a lot of optimization knowledge that was useful at one time but is now flat out wrong.

Oral culture is especially prone to this -- at least a blog post or a physical book has a date on it. You have no idea when the your co-worker's suggested optimization technique was developed.

I think 'theory' is just fine, you can't fix all communication failures by throwing moar words at it. At some point you have to make peace with the fact that not everybody uses words the same way you do.

"what's the word for this?"

I think the word (or phrase, rather) is "tribal knowledge".

tools of the trade

* from the book

secrets of the cult

I think there's a miscommunication here. By "write some efficient code" GP presumably meant the "pull" approach, in which you identify the code that's causing you performance problems and then try to make it faster, learning everything you need on the way. Rinse and repeat. It's a very efficient way to learn, and also good at teaching you those intutions that are hard to put down in writing.

That said, if anyone knows some kind of collected guide to writing efficient software, I'd be happy to learn about it, 'cause I haven't seen anything like this published.

I suppose the legendary "PC Game Programmers Encyclopedia" counts, although it is very old: http://qzx.com/pc-gpe/

There are various analyses of Doom and Quake source around which look at the techniques used there.

I am like you. Every big performance win i ever got was from changing algorithms. Sometimes i didn't write either the old or the new, but just swapped out a different implementation. A good handle on that stuff can get you a long way. And can certainly be picked up from reading. it's a lot closer to just math.

That said, spend a week with C and valgrind/cachegrind. There's a lot of theoretical stuff that is hard to get at (or is for me) without a little exposure to how the system works. a coupe hours here and there will extend your mental model to include the various layers of cache. That'll make the more esoteric stuff more accessible.

You could do worse than Steve McConnell's Code Complete - https://www.amazon.com/dp/0735619670.

I think Peter Norvig's article 'Teach Yourself Programming in Ten Years' [0], might be useful to you. One of his points is to 'Remember that there is a "computer" in "computer science"', along with examples of things to investigate.

[0] http://norvig.com/21-days.html

Efficiency can be done but it requires disciplines only taught in courses on real-time and embedded programming, and those seem to be fairly rare these days. Real-time and embedded programming teach the programmer how to reason about finite resources like time, energy, memory, and floating-point operations while requiring a level of mastery "down to the metal" that's also rare in overly abstraction-driven programming. RTE skills make every programmer better in the same way that glider skills make every pilot better.

I went to Cal and aside from some hardware classes i never had a real time or embedded class.

I think it requires an open mind to want to learn these things and put effort into it instead of blowing things off as premature or something the maintainer of the code behind you will come and fix.

Anecdotal but this matches my experience. I went to generic public school for ocean engineering and took tons of embedded classes. I then worked on a student robot project and was shocked when the CS students were completely lost with how to approach writing drivers and interfaces for the sensors and such. I thought that would be standard for CS.

> CS students were completely lost with how to approach writing drivers and interfaces for the sensors and such. I thought that would be standard for CS.

No, that would be standard for computer engineering, not CS.

Try to have a conversation about hardware to many practicing software developers who were CS majors and they'll look at you as if you've grown a second head.

I think that stuff tends to be more common for EEs and CompEs.

How should one learn real-time/embedded programming? Do you have any recommended readings or online resources?

I mostly agree, but I think you're too quick to assume that everyone who doesn't write optimal code is doing so because of some tech fad. Frequently it simply doesn't pay to write optimal code; there's no viable business case. Sure, using an array can be faster and not much harder than using a hash table, but writing a network protocol is frequently not worth the effort. In my experience, poor performance comes from the inability to make a business case (whether or not one actually exists), not from chasing fads.

I think good programmers, by default, try to write readable, easy-to-maintain code. If you ask them to write high-performance code then they will do it, knowing that readability might suffer because of it.

"Sometimes the excuse is that cpu and memory are cheap as if resources aren't under constant pressure from the hundreds of processes that all pretend they are the only one on the machine."

I find that this often extends beyond performance too. Many websites/apps waste lots of screen space because they were clearly designed for, and tested at, 1080p.

I think the secret is slightly different. Most think they can fully solve a problem. Such that their solution is the one everyone should use.

This manifests in assuming everyone can afford the abstraction boundaries chosen. Or that everyone needs the same tradeoffs. Or, ultimately, that everyone would come to the same conclusion.

I think of it as the desire of most people to boil software down to formulaic choices. Imagine if you came up with something as elegant as "F = ma" for software.

I feel this is somewhat related to the rise in functional programming. It is not that either of those things are bad goals to chase. But they are not ends to themselves.

The dirty secret in software development: most think they write efficient software, but very few know what to compare it to and how.

>The field is too in love with horribly inefficient frameworks. Writing network code and protocols is now considered too low level for people. Many are too caught up it what is tech-cool. They're like the holistic medicine practitioners of software and they don't even know it.

While the frameworks themselves are made for a particular purpose, they are often misused. One great example would be electron. I found an electron clone of Keepass. Now why would anybody want that.

And you find people justifying Huge bloated electron apps that take a minimum of 50 MB for a simple Hello World. And then they say memory is cheap. I say memory is cheap for 1 app at that size. Not running all apps at that size. If busybox can run GNU coreutils with a size less an MB then why can't they make more efficient apps?

Beats me.

>...then why can't they make more efficient apps?

They can.

>...frameworks... are often misused.

If you're not developing for a real-time system, like most (all?) larger games or software for embedded hardware, it is likely that you are not thinking about performance very much, even if you have the time and skill to do so.

You don't think about performance, as a developer, usually, until it becomes a problem, and there are so many layers and frameworks between an MP3 player and the CPU that this kind of thing should be expected.

It is much more difficult to keep a real-time system working in real-time than it is to keep something performing as well as a desktop computer user expects it to perform.

"Mechanical Sympathy"/empathy is a very good expression, kudos!

I wish I could take credit, but i think it was Martin Thompson that coined the phrase. I think it if a perfect deception of the, often subconscious, understanding of what is going on underneath the code.

> most think they how to write efficient software when they need to, but very few can

Doesn't mp3 transcoding have dedicated hardware instructions, making the job much easier?

It's not really "dedicated", it's just SIMD for FFT like most signal processing. Takes about 25 MIPS: https://retrocomputing.stackexchange.com/questions/2672/why-...

You can get dedicated MP3 low-power decoder chips. You might have one in a fancy PC soundcard (SBLive?), but I don't think it's included in baseline AC97.

(I remember having a 486DX that could decode MP3s with the Fraunhofer player at about 95% CPU but not Winamp, which was slighly too slow to keep up)

Does this also hold for modern smartphones?

I don't know exactly; the chipsets tend to be closed. I've seen baseband processors which have the AMR codec in hardware (adaptive multi-rate, used for GSM phone audio).

No, because it's so easy to begin with.

Satisfying a low-power constraint?