Try "Racing the Beam"; a neat book describing the programming challenges with writing software for the Atari 2600.

https://www.amazon.com/Racing-Beam-Computer-Platform-Studies...

Though, instead of reading, you really should just get over your complex and try writing some efficient code.

>get over your complex

It's not a complex. I'm always very surprised by this attitude. Theory serves practice.

Case and point: the OP mentions that an understanding of cache architecture enables you to reason about which data-structures exhibit good cache locality. Discovering that hashtables have poor cache locality by trial-and-error seems like a waste of time compared to gaining theoretical insights.

I'll second "Racing the Beam", although it's a history rather than a theoretical approach.

I think you may have hit a linguistic snag here; there's lots of "theory" in the classical CS sense dealing with performance from an O(n) point of view, such as Knuth's work. But for producing fast results on actual hardware you end up having to take into account lots of ugly details of the platform. Learning about cache behaviour doesn't really fit into CS so it's not the first thing people think of when you ask for theory.

"Knowledge transmitted by practitioners through written and oral culture outside of the academy": what's the word for this?

Part of the problem is that some of this stuff has a pretty short shelf life. There's a lot of optimization knowledge that was useful at one time but is now flat out wrong.

Oral culture is especially prone to this -- at least a blog post or a physical book has a date on it. You have no idea when the your co-worker's suggested optimization technique was developed.

I think 'theory' is just fine, you can't fix all communication failures by throwing moar words at it. At some point you have to make peace with the fact that not everybody uses words the same way you do.

"what's the word for this?"

I think the word (or phrase, rather) is "tribal knowledge".

tools of the trade

* from the book

secrets of the cult

I think there's a miscommunication here. By "write some efficient code" GP presumably meant the "pull" approach, in which you identify the code that's causing you performance problems and then try to make it faster, learning everything you need on the way. Rinse and repeat. It's a very efficient way to learn, and also good at teaching you those intutions that are hard to put down in writing.

That said, if anyone knows some kind of collected guide to writing efficient software, I'd be happy to learn about it, 'cause I haven't seen anything like this published.

I suppose the legendary "PC Game Programmers Encyclopedia" counts, although it is very old: http://qzx.com/pc-gpe/

There are various analyses of Doom and Quake source around which look at the techniques used there.

I am like you. Every big performance win i ever got was from changing algorithms. Sometimes i didn't write either the old or the new, but just swapped out a different implementation. A good handle on that stuff can get you a long way. And can certainly be picked up from reading. it's a lot closer to just math.

That said, spend a week with C and valgrind/cachegrind. There's a lot of theoretical stuff that is hard to get at (or is for me) without a little exposure to how the system works. a coupe hours here and there will extend your mental model to include the various layers of cache. That'll make the more esoteric stuff more accessible.

You could do worse than Steve McConnell's Code Complete - https://www.amazon.com/dp/0735619670.