Another one is to only look at memory allocations. Usually when there is a lot of allocations, a lot of data is being handled. This gives a bit of a different perspective where you start thinking of folding chunks of data together.
The nice thing about those two approaches is that they don't vary between CPU and architectures.
Cache invalidation can follow a similar misattribution pattern.
When you do succeed in making an improvement, the % of time for everything else should increase. 10% of 90% is 11%. Look back at the old results and see if anything else went up disproportionately, and see if you can understand why.
Related to that notion, and as a parting thought, since this is getting some love: game and A/V programmers understand something the rest of us do not. And that is a computation/time budget. It doesn’t matter if one thing takes four times as long as the other if they both take too long. You haven’t fixed “the problem” until you’ve fixed them all. Late is late.
There’s another “systematic” approach to making very large perf improvements that doesn’t fix problems from most to least significant but by functional unit. Fix all of the problems in that functional unit even if it’s 1% of the total (if you’re after a 5x speed gain it’s actually 5% of the total budget). If you make large scale changes you are going to have to do some expensive regression testing on this part of the code, tests or no tests. Once you’ve paid that price once, you will never go back for that last 2% because you can not justify it to your organization. You’re stuck with those in ten different functional areas, forever. What you can justify is boosting a raft of work from 18% to a 20% overall improvement.
I used 4 or 5 performance tools to look at the performance of Oil shell, and uftrace was one of the most important ones.
I hope to write a blog post about it soon. The overall results were very good -- Oil is now faster than bash despite doing more parsing by design -- though the biggest win was translating the code to C++ from Python :)
So it's unusual, but after translation, uftrace was a winner, and I used it largely by counting function calls and allocations, not measuring timing. Although you can use it in many different ways.
Oil's Parser is 160x to 200x Faster Than It Was 2 Years Ago
http://www.oilshell.org/blog/2020/01/parser-benchmarks.html
Another lesson is that aside from the big difference in execution speed, tools for measuring C++ performance are just way better than those for Python. Debugging tools are also better. Of course I still avoid C++ for most problems because of build times, headers, memory management, library style smorgasbord, etc.
Not perfect by any means but usually a nice check to verify optimization potential.
I learnt about this tool from Emery Berger's talk [2] on this (at strangeloop), which I highly recommend. Lots of really nice insight, even outside of this tool.
[1] https://github.com/plasma-umass/coz [2] https://www.youtube.com/watch?v=r-TLSBdHe1A
I found a lot of performance issues that others missed, and there were a number of quite notable cases where I had rather extraordinary results. Like 20% from tweaking (not even deleting) a method measured as taking 5% of run time. Not the weirdest I every did, but a solid number of top ten entries from this category.
How did I find them? Sorting by invocation count instead of run time. You can blindly jump into any of functions and get some useful results, but a more sophisticated approach is to think first about expected versus actual results. You think to yourself, “I just ran this task ten times. It has this much data to deal with. How many calls to this particular function should I have expected? 250? Then why was it called 759 times?”
What you find next will be architectural problems, or problems of data flow through the system (while some, including me, would argue that I just repeated myself, not everyone accepts that).
The fastest methods are the ones you never call, and I’m surprised at how frequently people ignore call count even though it’s often right there in the results. Maybe because it’s often the last value displayed, as if to say “This is barely worth mentioning.”