If you're lucky enough to eat your own dogfood, run unit tests against a copy(!) of your in-house database. A utility to anonymize the data was a fairly significant investment, but absolutely worth it in the long run. Being able to run and monitor benchmark unit tests for selected critical operations on enterprise-scale test data as part of the continuous-build process: fabulous!
How this typically works with performance is very similar, with a small team working to identify problematic areas where optimization would drive the highest impact, and the rest of the organization keeping performance in mind but not otherwise concerned with it in their day-to-day work.
If you can't recognize the percent where it matters a lot, clearly that percent does not exist and you're bothering about nothing.
I see a lot of talk about the importance of tuning Formula 1 cars in a world where everyone drives ford fiestas.
And when you do identify the 1%, you need to be testing optimizations with a profiler constantly while optimizing. Profile. Do some optimization. Profile again. Roll back if not successful. Repeat until done. It's impossible to optimize well if you're not doing profiling.
The ultimate tools would be either Intel's profiling tools, or ARM's profiling suite (both very expensive). But MSVC and GCC do a fantastic job of scheduling instructions to avoid pipeline stalls these days, so these deep profiling tools are unlikely to gain more than 2 or 3% performance increases these days. (Worthwhile pretty much only if you're writing GPU drivers for NVidia or AMD).
Taken as a given that 100% of your code is at least algorithmically correct in the first place. (Appropriately better than O(N^2) whenever possible).
- Former writer of graphics drivers, currently audio DSP engineer.
VTune allows you to determine where pipeline stalls are occurring at the instruction level (for that last 2 or 3% gain in performance). I haven't worked with ARM profilers (way out of my price range), but I assume, given the exorbitant price, they provide the same sort of in-depth analysis. Probably a handful of people on the planet that need that kind of in-depth analysis.
My point being, if most of the program is slowed down by a slew of wasted CPU cycles (costly abstractions, slow interpreted language…), there's a good chance what should have been obvious bottlenecks get drowned in a see of underperformance. They're harder to spot, and fixing them doesn't change much.
So before you even get to actual optimisation, your program should be fast enough that actual optimisations have a real impact. And yes, actual optimisation should be done quite rarely. But first, we need to make sure our programs aren't as slow as molasses. See https://www.youtube.com/watch?v=pgoetgxecw8
I feel you're missing the whole point.
It's immaterial whether anyone can get to optimizations that compound multiplicatively. The whole point is that halving something that costs nothing earns you nothing. That's the whole point. Go ahead and shave off that millisecond. Will anyone actually notice whether you add or remove that penalty? Odds are, not at all.
Monocypher's speed was actually an important component in its success in the embedded market, even though I didn't explicitly target it initially (I was lucky my portability driven decisions made it a good fit there).
There definitely are niches where there are quite a few performance optimization opportunities that users do care about.
In your example making something a user is actively waiting for go from 3 seconds to less than one is a great optimization target. What is not a great optimization target is making something the user is actively waiting for and that takes 30ms take 25ms instead. That's wasted money on developer time.
If your "user" is a developer of embedded software with memory constraints and using your library leaves them more room that's awesome. If your user was someone using the library on a general purpose computing device with loads of memory then the 2 vs. 5 does nothing.
I decided not put it off for later, and keep things simple for now.
For example, in the case of web development, if you build a medium-sized website with React (i.e. pretty normal behavior nowadays), then if you make default decisions that don't consider performance at all, your website will end up noticeably slow, because you will:
1. Write code that re-renders components all the time during loading and UI interactions,
2. Which depend on tons of third-party dependencies that perform poorly,
3. So you end up spending a ton of time in re-renders while the site loads and while someone is using it.
Dealing with this isn't literally the same performance work that Casey put in his article, because it's at a slightly higher level of abstraction, but it requires the same mindset. It requires writing most of your code (and taking on dependencies) with performance in mind, not just 1% of it. You can't avoid it without your notion of "clean code" including some amount of mechanical sympathy, rather than just being about abstract extensibility and generalization concerns.
This is a mistake that I see far too many government and utility sites making.
If you're really interested in the impact of performance issues on everyday life, you need to provide concrete examples instead of putting up unverifiable strawmen.
The truth of the matter is that 99% of real world applications run just fine, and it doesn't pay off to invest in shaving milliseconds here or there. Would it be desirable to have a magic wand to improve some edge cases? Yeah, why not? Is it worth to pay people to spend time with a stopwatch at hand to shave off these milliseconds? Not really. It's all about tradeoffs, and there is no real world payoff in wasting developers' time to shave off that millisecond here or there.
Not to mention that web apps redrawing everything whenever they feel like it can almost give motion sickness, lead to clicks on the wrong things etc.
Yeah, but users' time is worthless.
There's so much hardware out there that can run native applications just fine, that can play back HD video, that can run complex 3D real time video games, but crawl like molasses when loading your average webpage. Facebook and YouTube are terrible offenders, but so are your average blogs. Many banking websites are terrible (yet they don't have to be; my local credit union has a zippy website that looks attractive to boot, has modern design elements, etc.).
Maybe the hardware you're running is eye-wateringly fast, or maybe it's just barely fast enough and you don't need the cycles for anything else. But we're not talking milliseconds. We're talking order(s) of magnitude. I can't bring myself to believe you don't see at least some of it, if you just open your eyes and look around.
"Unverifiable!", he wrote from within a web browser.
Another app that I'm not sure if it's React or not is New Reddit. It is significantly slower on my computer and on a lot of people's computer and sometimes you have to refresh the page because it consumes too much memory.
I can come up with other local examples, from Germany. Vatenfall's website seems to "traditional" page navigation, but the content is loaded via a framework. Due to having to reinitialize everything, text takes up to 5 seconds to appear when using the back button or when navigating for page to page. Similar things happen in the German Agentür fur Arbeit.
Let's not even get into low end phones in developing countries or bargain basement android tablets, let's stick with something straightforward - an ordinary PC.
I took a quick look online, sorting by cheapest first I found something with an AMD 3015e in. Based on cpubenchmark.net that gets a benchmark of 2691. Taking a look at the big list of CPUs I see that's equivalent to a powerful desktop CPU from 2008, or a decent laptop from 2012. (The Apple M2 in the current Macbook Air gets a score of 15369, just for comparison.)
So, if you're writing PC software or making a fancy web app and you want everyone to have a good experience with it then you should see how it runs on a terrible new laptop, or a 12 year old good laptop, or a 15 year old powerful desktop.
(And yeah we all have SSDs now which is much better than in the old days, and JS is generally single thread, and single threaded CPU performance has not improved so much - but I think my point still stands.)
You're taking deep quaffs of the Kool-Aid and so are most of the people commenting on this story. General software responsiveness and reliability (i.e., usefulness) has been in decline for decades. This is an objective fact.
Writing objective reality off as mere "milliseconds", "edge-cases", or only relevant for "toy problems" exemplifies the arrogance and severe incompetence of most programmers. People are seriously trying to talk down to Casey Muratori when in all likelihood they haven't accomplished even 1% as much as him as programmers.
I get it—no one wants to leave fantasy-land as long as the easy money is flowing. But sooner or later the glittering carriage turns back into a pumpkin.
It's certainly a huge problem for real world applications
Who was talking about a single millisecond here?
I notice, broadly, two types of people who engage in these arguments.
1> OMG, computers are thousands of times faster than they were a decade ago, why is everything not lightning fast? Why are so many things slower than they were back then? Why is my chat program eating 2GB(!) of RAM?
2> Because we're busy writing six billion features on our Nth iteration of this problem space, we can't be bothered to shave a few millis bro!
And they just talk past each other.
You attempted to present an argument that's a textbook example of an hyperbolic fallacy.
There are worlds of difference between "this code does not sit in a hot path" and "let's spend multiple seconds of local processing time".
This blend of specious reasoning is the reason why the first rule of software optimization is "don't". Proponents of mindlessly going about the 1% edge cases fail to understand that the whole world is comprised of the 99% of cases where shaving off that millisecond buys you absolutely nothing, with a tradeoff of producing unmaintainable code.
The truth of the matter is that in 99% of the cases there is absolutely no good reason to run after these relatively large performance improvements if in the end the user notices absolutely nothing. Moreso if you're writing async code that stays far away from any sort of hot path.