I've solved a lot of arguments with godbolt and simple performance tests. Some topics are recurring themes among software engineers e.g.:
- compilers are almost always better at micro-optimizations than you are
- disk I/O is almost never a bottleneck in competent designs
- brute-force sequential scans are often optimal algorithms
- memory is best treated as a block device
- vectorization can offer large performance gains
- etc...
No one is immune to this. I am sometimes surprised at the extent to which assumptions are no longer true when I revisit optimization work I did 10+ years ago.
Most performance these days is architectural, so getting the initial design right often has a bigger impact than micro-optimizations and localized Big-O tweaks. You can always go back and tweak algorithms or codegen later but architecture is permanent.
When I'm building stuff I try my best to focus on "correctness", and try to come up with an algorithm/design that will encompass all realistic use cases. If I focus on that, it's relatively easy to go back and convert my `decimal` type to a float64, or even convert an if statement into a switch if it's actually faster.
(the techniques that used to work were similar to earlier Java versions and overall very dynamic languages with some exceptions, the techniques that still work and now are required today are the same as in C++ or Rust)
There are a few articles on msft devblogs that cover from-netframework migration to older versions (Core 3.1, 5/6/7):
- https://devblogs.microsoft.com/dotnet/bing-ads-campaign-plat...
- https://devblogs.microsoft.com/dotnet/microsoft-graph-dotnet...
- https://devblogs.microsoft.com/dotnet/the-azure-cosmos-db-jo...
- https://devblogs.microsoft.com/dotnet/one-service-journey-to...
- https://devblogs.microsoft.com/dotnet/microsoft-commerce-dot...
The tl;dr is depending on codebase the latency reduction was anywhere from 2x to 6x, varying per percentile, or the RPS was maintained with CPU usage dropping by ~2-6x.
Now, these are codebases of likely above average quality.
If you consider that moving 6 -> 8 yields another up to 15-30% on average through improved and enabled by default DynamicPGO, and if you also consider that the average codebase is of worse quality than whatever msft has, meaning that DPGO-reliant optimizations scale way better, it is not difficult to see the 10x number.
Keep in mind that while particular regular piece of enterprise code could have improved within bounds of "poor netfx codegen" -> "not far from LLVM with FLTO and PGO", the bottlenecks have changed significantly where previously they could have been in lock contention (within GC or user code), object allocation, object memory copying, e.g. for financial domains - anything including possibly complex Regex queries on imported payment reports (these alone have now difference anywhere between 2 and >1000[0]), and for pretty much every code base also in interface/virtual dispatch for layers upon layers of "clean architecture" solutions.
The vast majority of performance improvements (both compiler+gc and CoreLib+frameworks), which is difficult to think about, given it was 8 years, address the above first and foremost. At my previous employer the migration from NETFX 4.6 to .NET Core 3.1, while also deploying to much more constrained container images compared to beefy Windows Server hosts, reduced latency of most requests by the same factor of >5x (certain request type went from 2s to 350ms). It was my first wow moment when I decided to stay with .NET rather than move over to Go back then (was never a fan of syntax though, and other issues, which subsequently got fixed in .NET, that Go still has, are not tolerable for me).
[0] Cumulative of
https://devblogs.microsoft.com/dotnet/regex-performance-impr...
https://devblogs.microsoft.com/dotnet/regular-expression-imp...
https://devblogs.microsoft.com/dotnet/performance-improvemen...
All of the 6x performance improvement cases seem to be related to using the .net based Kestrel web server instead of IIS web server, which requires marshalling and interprocess communication. Several of the 2x gains appear to be related to using a different database backend. Claims that regex performance has improved a thousand-fold.... seem more troubling than cause for celebration. Were you not precompiling your regex's in the older code? That would be a bug.
Somewhere in there, there might be 30% improvements in .net codegen (it's hard to tell). Profile Guided Optimization (PGO) seems to provide a 35% performance improvement over older versions of .net with PGO disabled. But that's dishonest. PGO was around long before .net Core. And claiming that PGO will provide 10x performance because our code is worse than Microsoft's code insults both our code and our intelligence.
With a Roslyn-based compiler at work I saw 20 % perf improvement just by switching from .NET Core 3.1 to .NET 6. No idea how slow .NET Framework was, though. I probably can't target the code to that anymore.
But for regex even with precompilation, the compiler got a lot better at transforming the regex into an equivalent regex that performs better (automatic atomic grouping to reduce unnecessary backtracking when it's statically known that backtracking won't create more matches for example) and it also benefits a lot from the various vectorized implementations of Index of, etc. Typically with each improvement of one of those core methods for searching stuff in memory there's a corresponding change that uses them in regex.
So where in .NET Framework a regex might walk through a whole string character by character multiple times with backtracking it might be replaced with effectively an EndsWith and LastIndexOfAny call in newer versions.
(the distinction becomes important for targets serviced by Mono, so to outline the difference Mono is usually specified, while CoreCLR and RyuJIT may not be, it also doesn't help that JIT, that is, the IL to machine code compiler, also services NativeAOT, so it gets more annoying to be accurate in a conversation without saying the generic ".net compiler", some people refer to it as JIT/ILC)
And indeed, on the C# -> IL side there's little that's being actually optimized. Besides collection literals there's also switch statements/expressions over strings, along with certain pattern matching constructs that get improved on that side.
Is it a public project?
Also, IIS hosting through Http.sys is still an option that sees separate set of improvements, but that's not relevant in most situations given the move to .NET 8 from Framework usually also involves replacing Windows Server host with a Linux container (though it works perfectly fine on Windows as well).
On Regex, compiled and now source generated automata has seen a lot of work in all recent releases, it is night and day to what it was before - just read the articles. Previously linear scans against heavy internal data structures (matching by hashset) and heavy transient allocations got replaced with bloom-filter style SIMD search and other state of the art text search algorithms[0], on a completely opposite end of a performance spectrum.
So when you have compiler improvements multiplied by changes to CoreLib internals multiplied by changes to frameworks built on top - it's achievable with relative ease. .NET Framework, while performing adequately, was still that slow compared to what we got today.
[0] https://github.com/dotnet/runtime/tree/main/src/libraries/Sy...
And you have misrepresented the contents of the blogs. The projects discussed in the blogs are typically claiming ~30% improvements (perhaps because they weren't using static PGO in their 4.7.0 incarnation), with two dramatic outliers that seem to be related to migrating from IIS to Kestrel.
It’s also convenient to ignore the rest of the content at the links but it seems you’re more interested in proving your argument so the data I provided doesn’t matter.
> Were you not precompiling your regex's in the older code? That would be a bug.
I never heard of this before. Perl has legendary fast regexen and I never heard of this feature. Does Java do it? I don't think so, and the regexes are fast enough in my experience. Can you name a language when regexen are precompiled?But I agree, algorithmic complexity is generally the only thing I focus on, and even then it's almost always a case of "will that actually matter?" If I know that `n` is never going to be more than like `10`, I might not bother trying to optimize an O(n^2) operation.
What I feel often gets ignored in these conversations is latency; people obsess over some "optimization" they learned in college a decade ago, and ignore the 200 HTTP or Redis calls being made ten lines below, despite the fact that the latter will have a substantially higher impact on performance.
My experience is the opposite - a sizeable chain of ifs has more that can go wrong precisely because it is more flexible. If I'm looking at a switch, I immediately know, for instance, that none of the tests modifies anything.
Meanwhile, while a missing break can be a brutal error in a language that allows it, it's usually trivial to set up linting to require either an explicit break or a comment indicating fallthrough.
It turns out the algorithmic complexity of a switch statement and the equivalent series of if-statements is identical. The bijective mapping between them is close to the identity function. Does a naive compiler exist that doesn't emit the same instructions for both, at least outside of toy hobby project compilers written by amateurs with no experience?
If statements are unbounded, unconstrained logic constructs, whereas switch statements are type-checkable. The concern about missing break statements here is irrelevant, where your linter/compiler can warn about missing switch cases they can easily warn about non-terminated (non-explicitly marked as fall-through) cases.
For non-compiled languages (so branch prediction is not possible because the code is not even loaded), switch statements also provide a speed-up, i.e. the parser can immediately evaluate the branch to execute vs being forced to evaluate intermediate steps (and the conditions to each if statement can produce side-effects e.g. if(checkAndDo()) { ... } else if (checkAndDoB()) { ... } else if (checkAndDoC()) { ... }
Which, of course, is a potential use of if statements that switches cannot use (although side-effects are usually bad, if you listened to your CS profs)... And again a sort of "static analysis" guarantee that switches can provide that if statements cannot.
Awkward switch syntax aside, the switch is simpler to reason about. Fundamentally we should strive to keep our code simple to understand and verify, not worry about compiler optimizations (on the first pass).
That said, the linear test is often faster due to CPU caches, which is why JITs will often convert switches to if/elses.
IMO, switch is clearer in general and potentially faster (at very least the same speed) so it should be preferred when dealing with 3+ if/elseif statements.
if statements are dumber, and maybe arguably uglier, but I feel like they're also more clear, and people don't try and be clever with them.
For example, with java there's enhanced switch that looks like this
var val = switch(foo) {
case 1, 2, 3 -> bar;
case 4 -> baz;
default -> {
yield bat();
}
}
The C style switch break stuff is definitely a language mistake.However most languages have pretty permissive switch statements just like C.
So no offense, but I would revisit the wider world of language constructs before claiming that switch statements are "all bad". There are plenty of bad languages or languages with poor implementations of syntax, that do not make the fundamental language construct bad.
var len = slice switch
{
null => 0,
"Hello" or "World" => 1,
['@', ..var tags] => tags.Length,
['{', ..var body, '}'] => body.Length,
_ => slice.Length,
};
(it supports a lot more patterns but that wouldn't fit)However, the moment you add a side effect or something more complicated like a method call, it becomes really hard for the complier to know if that sort of optimization is safe to do.
The benefit of the switch statement is that it's already well positioned for the compiler to optimize as it does not have the "you must run these evaluations in order" requirement. It forces you to write code that is fairly compiler friendly.
All that said, probably a waste of time debating :D. Ideally you have profiled your code and the profiler has told you "this is the slow block" before you get to the point of worrying about how to make it faster.
This is not entirely true either... Measure. There are many cases where the optimiser will vectorise a certian algorithm but not another... In many cases On^2 vectorised may be significantly faster than On or Onlogn even for very large datasets depending on your data...
Make your algorithms generic and it won't matter which one you use, if you find that one is slower swap it for the quicker one. Depending on CPU arch and compiler optimisations the fastest algorithm may actually change multiple times in a codebases lifetime even if the usage pattern doesn't change at all.
Reminds me of the classic https://stackoverflow.com/questions/24848359/which-is-faster...
When I was taught about performance, it was all about benchmarking and profiling. I never needed to trust what my professors taught, because they taught me to dig in and find the truth for myself. This was taught alongside the big-O stuff, with several examples where "fast" algorithms are slower on small inputs.
JVM ecosystem has IntelliJ Idea profiler and similar advanced tools (AFAIK).
.NET has VS/Rider/dotnet-trace profilers (they are very detailed) to produce flamegraphs.
Then there are native profilers which can work with any AOT compiled language that produces canonically symbolicated binaries: Rust, C#/F#(AOT mode), Go, Swift, C++, etc.
For example, you can do `samply record ./some_binary`[0] and then explore multi-threaded flamegraph once completed (I use it to profile C#, it's more convenient than dotTrace for preliminary perf work and is usually more than sufficient).
Also there is learning curve to grouping and aggregating data.
that said, c++ is usually a language you use when you care about performance, at least to an extent. it's worth understanding features like nrvo and rewriting functions to allow the compiler to pick the optimization if it doesn't hurt readability too much.
Writing well structured readable code is typically far more important than making it twice as fast. And those times can rarely be predicted beforehand, so you should mostly not worry about it until you see real performance problems.
Why should you care about performance?
I can give you my personal experience: I’ve been working on a Java web/application server for the past 15 years and a typical request (only reading, not writing to the db) would take maybe 4-5 ms to execute. That includes HTTP request parsing, JSON parsing, session validation, method execution, JSON serialization, and HTTP response dispatch. Over the past 9 months I have refactored the entire application for performance and a typical request now takes about 0.25 ms or 250 microseconds. The computer is doing so much less work to accomplish the same tasks, it’s almost silly how much work it was doing before. And the result is the machine can handle 20x more requests in the same amount of time. If it could handle 200 requests per second per core before, now it can handle 4000. That means the need to scale is felt 20x less intensely, which means less complexity around scaling.
High performance means reduced scaling requirements.
I'm not saying you completely throw caution to the wind, I'm just saying that there's a finite amount of human resources and it can really vary how you want to allocate them. Sometimes the better path is to just throw money at the problem.
It really depends.
I personally don’t like the idea of throwing compute at a slow solution. I like when the extra effort has been put into something. The good feeling I get from interacting with something that is optimal or excellent is an end in itself and one of the things I live for.
CPUs are ridiculously fast now, and compilers are really really good now too. I'm not going to say that processing speed is a "solved" problem, but I am going to say that in a lot of performance-related cases the CPU processing is probably not your problem. I will admit that this kind of pokes holes in my previous response, because introducing more machines into the mix will almost certainly increase latency, but I think it more or less holds depending on context.
But I think it really is a matter of nuance, which you hinted at. If I'm making an admin screen that's going to have like a dozen users max, then a slow, crappy solution is probably fine; the requests will be served fast enough to where no one will notice anyway, and you can probably even get away with the cheapest machine/VM. If I'm making an FPS game that has 100,000 concurrent users, then it almost certainly will be beneficial to squeeze out as much performance out of the machine as possible, both CPU and latency-wise.
But as I keep repeating everywhere, you have to measure. You cannot assume that your intuition is going to be right, particularly at-scale.
(Ironically, HN is buckling under load right now, or some other issue.)
If your problem can fit on one server, it can massively reduce engineering and infrastructure costs.
For example, much to the annoyance of a lot of people, I don't typically use floating point numbers when I start out. I will use the "decimal" or "money" types of the language, or GMP if I'm using C. When I do that, I can be sure that I won't have to worry about any kind of funky overflow issues or bizarre rounding problems. There might be a performance overhead associated with it, but then I have to ask myself "how often is this actually called?"
If the answer is "a billion times" or "once in every iteration of the event loop" or something, then I will probably eventually go back and figure out if I can use a float or convert it to an integer-based thing, but in a lot of cases the answer is "like ten or twenty times", and at that point I'm not even 100% sure it would be even measurable to change to the "faster" implementations.
What annoys me is that people will act like they really care about speed, do all these annoying micro-optimizations, and then forget that pretty much all of them get wiped out immediately upon hitting the network, since the latency associated with that is obscene.
It is why many language ecosystems suffered from performance issues for a really long time even if completely unwarranted.
Is changing ifs to switch or vice versa, as outlined in the post above, a waste of time? Yes, unless you are writing some encoding algorithm or a parser, it will not matter. The compiler will lower trivial statements to the same codegen and it will not impact the resulting performance anyway even if there was difference given a problem the code was solving.
However, there are things that do cost like interface spam, abusing lambdas writing needlessly complex wokflow-style patterns (which are also less readable and worse in 8 out of 10 instances), not caching objects that always have the same value, etc.
These kinds of issues, for example, plagued .NET ecosystem until more recent culture shift where it started to be cool once again to focus on performance. It wasn't being helped by the notion of "well-structured code" being just idiotic "clean architecture" and "GoF patterns" style dogma applied to smallest applications and simplest of business domains.
(it is also the reason why picking slow languages in general is a really bad idea - everything costs more and you have way less leeway for no productivity win - Ruby and Python, and JS with Node.js are less productive to write in than C#/F#, Kotlin/Java or Go(under some conditions))
There are plenty of cases where even the "slow" implementation is more than fast enough, and there are also plenty of cases where the "correct" solution (from a big-O or intuition perspective) is actually slower than the dumb case. Intuition helps, you have to measure and/or look at the compiled results if you want to ensure correct numbers.
An example that really annoys me is how every whiteboard interview ends up being "interesting ways to use a hashmap", which isn't inherently an issue, but they will usually be so small-scoped that an iterative "array of pairs" might actually be cheaper than paying the up-front cost of hashing and potentially dealing with collisions. Interviews almost always ignore constant factors, and that's fair enough, but in reality constant factors can matter, and we're training future employees to ignore that.
I'll say it again: as far as I can tell, you have to measure if you want to know if your result is "faster". "Measuring" might involve memory profilers, or dumb timers, or a mixture of both. Gut instincts are often wrong.
When talking about not assuming optimizations...
32bit float is slower than 64bit float on reasonable modern x86-64.
The reason is that 32bit float is emulated by using 64bit.
Of course if you have several floats you need to optimize against cache.
x86-64 requires the hardware to support SSE2, which has native single-precision and double-precision instructions for floating-point (e.g., scalar multiply is MULSS and MULSD, respectively). Both the single precision and the double precision instructions will take the same time, except for DIVSS/DIVSD, where the 32-bit float version is slightly faster (about 2 cycles latency faster, and reciprocal throughput of 3 versus 5 per Agner's tables).
You might be thinking of x87 floating-point units, where all arithmetic is done internally using 80-bit floating-point types. But all x86 chips in like the last 20 years have had SSE units--which are faster anyways. Even in the days when it was the major floating-point units, it wasn't any slower, since all floating-point operations took the same time independent of format. It might be slower if you insisted that code compilation strictly follow IEEE 754 rules, but the solution everybody did was to not do that and that's why things like Java's strictfp or C's FLT_EVAL_METHOD were born. Even in that case, however, 32-bit floats would likely be faster than 64-bit for the simple fact that 32-bit floats can safely be emulated in 80-bit without fear of double rounding but 64-bit floats cannot.
[0] https://gist.github.com/dosshell/495680f0f768ae84a106eb054f2...
Sorry for the confusion and spreading false information.
SIMD/MIMD will benefit of working on smaller width. This is not only true because they do more work per clock but because memory is slow. Super slow compared to the cpu. Optimization is alot about cache misses optimization.
(But remember that the cache line is 64 bytes, so reading a single value smaller than that will take the same time. So it does not matter in theory when comparing one f32 against one f64)
Yeah, that's never been true. Old compilers would often compile a switch to __slower__ code because they'd tend to always go to a jump table implementation.
A better reason to use the switch is because it's better style in C-like languages. Using an if statement for that sort of thing looks like Python; it makes the code harder to maintain.
(Also, Python has a switch-like construct now.)
PS: I'm presently revisiting C++14 because it's the most universal statically-compiled language to quickly answer interview problems. It would be unfair to impose Rust, Go, Elixir, or Haskell on an interviewer software engineer.
Very true, though there is one case where one can be highly confident that this is the case: code elimination.
You can't get any faster than not doing something in the first place.
Quick example, I got in an argument with someone a few years ago that claimed in C# that a `switch` was better than an `if(x==1) elseif(x==2)...` because switch was "faster" and rejected my PR. I mentioned that that doesn't appear to be true, we went back and forth until I did a compile-then-decompile of a minimal test with equality-based-ifs, and showed that the compiler actually converts equality-based-ifs to `switch` behind the scenes. The guy accepted my PR after that.
But there's tons of this stuff like this in CS, and I kind of blame professors for a lot of it [1]. A large part of becoming a decent engineer [2] for me was learning to stop trusting what professors taught me in college. Most of what they said was fine, but you can't assume that; what they tell you could be out of date, or simply never correct to begin with, and as far as I can tell you have to always test these things.
It doesn't help that a lot of these "it's faster" arguments are often reductive because they only are faster in extremely minimal tests. Sometimes a microbenchmark will show that something is faster, and there's value in that, but I think it's important that that can also be a small percentage of the total program; compilers are obscenely good at optimizing nowadays, it can be difficult to determine when something will be optimized, and your assertion that something is "faster" might not actually be true in a non-trivial program.
This is why I don't really like doing any kind of major optimizations before the program actually works. I try to keep the program in a reasonable Big-O and I try and minimize network calls cuz of latency, but I don't bother with any kind of micro-optimizations in the first draft. I don't mess with bitwise, I don't concern myself on which version of a particular data structure is a millisecond faster, I don't focus too much on whether I can get away with a smaller sized float, etc. Once I know that the program is correct, then I benchmark to see if any kind of micro-optimizations will actually matter, and often they really don't.
[1] That includes me up to about a year ago.
[2] At least I like to pretend I am.