I deplore that - its side-effects writ large. But those that use it, find it reasonable and sensible.
It's become hard to distinguish right and wrong when it comes to CPU optimization. Different versions of the 'same' CPU can have wildly different sweet spots.
Indeed, though I am shocked when people post benchmarks in which they are obviously unaware of this issue. It's why compilers have all those architecture switches.
If I understand your comment correctly: in the case of the posted paper, the compiler optimization issue the author mentioned was a semantics issue, not an architecture issue. If your comment was about optimizing for the multiprocessor case, I apologize.
Oh for the good ol days when register size was all you had to think about!
The problem isn’t that C++ exceptions are slow when used the way the compiler expects. The problem is a mismatch between how C++ expects you to use the feature, and how people are actually using the feature.
I can’t find the story now, but I read about this happening with Ruby on Rails. Someone dug into why rails was so slow at Twitter, and found the way rails was looping through an array was that it would loop unbounded, and catch and discard the array out of bounds exception at the end of the loop. Whenever Ruby throws, it allocates 10k of memory and fills it with all sorts of information for debugging - including a full stack trace. And it was doing this work in the background every time someone iterated through an array. Fixing this resulted in a massive speed up at Twitter, and presumably across the entire rails ecosystem.
I saw the same thing at (big tech company) a decade or so ago. I was working on a project which used GWT. We brought some people in to help us optimize, because the program was too slow. The first thing they found was that the JS VM was spending most of its time in the exception handler for some reason. Turned out one of our engineers had a habit of using throw & catch as a way to do multi level returns in complex code in Java. But the exception was being converted to a javascript exception by GWT. And javascript exceptions are (were?) super slow.
Y’all gotta stop using exceptions like that. I know it feels clever. But in most programming languages, using exceptions for control flow will kill your performance.
Jesus
Great detail I'd forgotten: At the time twitter took 460ms of compute to process each request. Almost all the CPU time was in bcopy, constructing the big string backtraces for exceptions (that were then being immediately discarded).
I think the approach Go, Rust and others take there is the right way to go. By making panics annoying or even impossible to recover from in some cases, they ensure that they are only used for their intended purpose and are not abused as a general purpose control flow mechanism.
We can't just blame the programming language for programmers not understanding how computers work.
I think as computers have gotten faster, and languages higher level, we've stopped talking about computers as mechanical devices. And this is a really important perspective to have.
Can you answer these questions about your program?
- How big is your binary / JS bundle? What parts take up most of the space?
- When your program runs, what does the computer spend most of its time executing? What parts of your program are the slowest, and why?
- How big is the memory footprint? Which parts of your program use the most RAM?
For binary programs (like rust / Go / C), which patterns are easier or harder for the optimizer?
If there are two ways to design your code, how do you discover which approach will run faster?
This stuff shouldn't be considered advanced concepts. An architect understands the building they've designed. A chef knows what their food tastes like. When you program, you're making a thing. You should understand what you made and how it will be executed on the computer.
shudder The worst of cargo cults!
A design pattern is a way of communication. If you turn it into a procrustean tool you don't even understand why they exist.
It is usually better if it can be captured in a library. But sometimes, particularly in weak languages, nothing but core language support will do. A fancy core language feature should embarrass the language designer.
Sometimes, a language evolves to the point where a core feature could be as well implemented as a library component. That is a marker of real progress, because now the feature can be offered in a dozen variations in a dozen libraries, each tailored to local conditions, where the core feature had to ruthlessly ignore all but the most generic use cases.
Imagine writing a parser. Can you use exceptions? It's hard to say! In fact, often impossible to say! How often an exception is thrown is highly dependent on the input data. If you're in a controlled environment with nearly always well formed data, it might be a perf win! But there's a ton of complexity in just deciding which type of error to use, and thats on top of the complexity of now having multiple ways of returning errors!
No one likes code that has 2+ ways of indicating method failure. If you are a library author, do your users need to check for both Exceptions and Error codes? If you are a nice author, you'll just choose 1 way to indicate failure and wrap where necessary.
If you want to use exceptions, they should be used in very limited scope when perf measurements indicate they would help, tightly wrapped in a catch, and then immediately converted to some other standard error type.
Even better, we should just let the choice of when to use exceptions up to an optimizing compiler, and provide the compiler a function to convert between exceptions and our more general purpose error type of choice.
I am aware there are functional languages that have other ways of dealing with this, or languages with things like optional values etc. But fundamentaly I like the idea that there's just a mechanism to just collapse, and collapse in a way that is traceable.
Having said that, my experience essentially does not involve catching exceptions, to me that's what is almost unthinkable (read: probably a bad hack). I guess my use case is not performance but separateing normal code function from catastrophic errors. Could you explain, do you think this is a fundamental mistake in the way I'm approaching this?
Which is part of the problem. If you're not catching the exception, you don't need exceptions. You just need a way to log stuff, such as a trace, and exit the program. New languages such as Rust and Go encourage this explicitly - that is, use the ordained error path in the return value for expected errors, and only panic for catastrophic errors (or use panic in hotpath where return overhead has unacceptable performance impact and errors are super rare).
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p070...
> Programs bugs are not recoverable run-time errors and so should not be reported as exceptions or error codes. — We must express preconditions, but using a tool other than exceptions... migrate std:: away from throwing exceptions for precondition violations.
Exactly. But the upstream caller of your function might need it.
And that’s the beauty of it. If I don’t know how to handle a particular error in one context, I can just leave it and let it propagate upwards.
For example downloadFile()->parseHttpResponseStrean()->readSocket()
If readSocket throws a timeoutError I don’t need every http library to explicitly deal with it. What can they even do about it, apart from retrying? The application level code calling the downloadFile function is still free to catch the error and gracefully show an error to the user or use a default file from disk if they please.
auto fileOrError = downloadFile();
if (fileOrError is error) {...} if err != nil {
…
}
On top of this it still isn’t bullet proof. There are infinite number of errors that can happen almost anywhere, especially when IO is involved; Disconnects, Time-outs, StackOverflow, ArrayOutOfBounds, DivideByZero, Overflow, NoSuchKeyInDictionary. And so on and so on.
Going back to go, Many low quality libraries still use panic, which gives you two ways to handle errors, one which is considered exotic so you don’t normally put recover() in place, then get bitten at worst moment.
Another example being Java where checked exceptions force beginners to handle errors, where most people don’t know what to do about it, so they just log.error() and return a dummy value, giving no way for upstream to detect such error and causing the program to blow up later instead. That said, checked exceptions is probably the best middle ground between verbosity and safety. It’s just greatly misunderstood.
Very few languages are as finely tuned to the needs of libraries and users of libraries as C++. Exceptions are an essential part of that compact. Failing to abide by it makes your library a bad citizen.
I always recommend reading Joe Duffy's post on errors for the curious.
Yes + syntactic sugar. Propagating an error up can and should be a single character of code.
> collapse in a way that is traceable.
Zig supports "error traces" (note, not stack traces) by default, since they have a single common way of reporting errors and a smart compiler.
> I like the idea that there's just a mechanism to just collapse,
Go uses Panic + Recover.
The overlap between "things that fail and the caller knows something useful to do", and "things that fail and somebody maybe six levels back in the call chain might know something to do" is remarkably thin. Most things are the latter. Once in a while, it makes sense to provide one for each: (1) open a file, and it had better succeed or you might have to bail out, or (2) see if a file is there.
The reason exception is the default, aside from that there are plenty of ways to return an error report, is that when you need code to clean up, the destructors are right there. They get exercised every time through the code, so are well-tested. How often is your special-snowflake error handler ever exercised? And the next one? And the next one? Even before you have given a moment's thought to error handling, your destructors are cleaning up, correctly. So, even if you haven't written code to handle it, your program has remained in a well-defined state. And, very often, you don't even need to write them: they are auto-generated from things the compiler already knows.
If you have set up a high-level catch block, you get there without fuss, and that code knows something correct to do, even if it is only to shut down cleanly. If you left a record of what was about to be attempted, that code knows what to try next.
My point is exactly that. It's more important to have a single common way of doing error handling than it is to have a small perf improvement. That's why I recommend return values. You'll never hit a perf problem with them.
Exceptions cannot be used in code hot paths (~thousands of iterations) when it's not a small perf problem anymore. Therefore, they cannot be your only method for signaling failure. And therefore, assuming we care about consistency and composability of code, it's far better to use return value based error handling unless you have good reason (almost never). Ideally with syntatic sugar from the compiler (see Rust, Zig, Swift, etc).
> when you need code to clean up, the destructors are right there.
Exceptions are not needed to achieve this. C++ calls the same 'well tested' destructors when you return from a function.
The one point in your favor, that I acknowledged above, is that C++ requires exceptions to report failure from Constructors and Destructors, for OOP, std::vector initialization, for much of the standard library etc. Thats a travesty.
You are arguing that programmers should not be obliged to make judicious choices. But that is exactly the job of every programmer. Dodging consequential choices practically defines a bad programmer.
Your assertion that destructors handle errors reported by return value is disingenuous, if not actively dishonest. The code checking for and acting on a failure is not in a destructor, and is never exercised except when you succeed in provoking exactly that failure.
You are writing bad code, and should be ashamed.
The only call where if's should probably be minimized is on hot paths, where there shouldn't be any I/O or other exception code anyway.
I would agree that manual stack unwinding (many stack frames deep) by propating error values up the call chain is bad. The problem here is not the if's, but that the program structure is unnecessarily convoluted. This can happen when specific code is calling generic code a lot (which can be avoided by having the generic code call the specific code instead).
> You are writing bad code, and should be ashamed.
Woah, calm your horses!
The overhead of doing this is measured in the article itself.
Program structure corruption is another problem. They add.
Bad code is a tax on all of us.
I don't know, as a C programmer, somehow I rarely run into these situations where I have to check error return values. I think the reason is that I work hard to avoid wrappers around wrappers around wrappers. Mostly error checking is required when interacting with the OS - i.e. for I/O, which is not critical paths.
A simple example would be memory allocation. A good C programmer allocates memory upfront. A bad C++ programmer could go, "eh, if it's so convenient I'll simply declare this std::vector locally, and if allocation fails it'll throw an exception and RAII and exceptions will solve the error handling issue magically without me having to type a single keystroke". Of course allocating each time is a lot slower than allocating only a single time upfront; no matter how much faster or convenient and individual allocation and error checking would be. This example shows how the perception of speed can often be warped because we're measuring the wrong thing entirely.
What is bad code is often not clear cut, and neither is how to improve it.
Comparing a good C coder to a bad C++ coder is meaningless: By definition, a bad coder makes bad choices, whatever the language.
Bad code is slow in a place where speed matters. It is improved by making it fast. Language choice does not affect this.
What language choice does affect is whether you can afford to spend the attention needed to make the code fast. C++ provides tools to offload busywork, freeing your attention to apply to what matters. It remains the programmer's job to choose that.
I still can't imagine a situation where speed matters and where exceptional situations are likely and also hard to handle in a performant way with explicit error checking. Do you have any examples?
It is true that in C++, as in Rust, it is much more common than in C to have very short functions that call other very short functions that, all composed together, do a job that, absent abstraction, might all be coded in a single larger function tailored to the specific use case; and the compiler squeezes out most of the calls and generates object code as if for that single, longer function. Therefore, there are many more places where an error return report would need to bubble up through, and that would cripple performance to handle the naive way, without exceptions.
It might be that some languages that have a native "result" type, and automatically generate checking code at the call site, can squeeze out most of the intermediate-level checks when composing inline functions together, e.g. if actual errors mostly occur at leaf nodes of the call tree. That could mitigate the heavy overhead cost of the method. But I don't know if any compilers do such an optimization, or if they do, how well it really works. Coding an optimizer is hard.
Do you have any evidence for these claims? If a called function is quite long by itself, then there is hardly any added overhead if the client checks the returned code and that check would be "redundant".
For example, if I make a syscall and check the return value, there will be a duplicated checking effort with the kernel code in a way - but compared to the cost of calling into the kernel that overhead is a negligible cost for the benefit of modularization (kernel vs userspace application). This overhead will be very close to unmeasureable, and does not justify the introduction of exceptions which are an additional mechanism to return values which introduce syntactic and binary incompatibilities (i.e. non-orthogonal functionality).
Oh, by the way - wouldn't exceptions and stack unwinding have to do just as much work per stack frame? It can't really just skip over all frames in a single instruction.
I think the best solution to cruft is generally to not have it in the first place - instead of introducing a mechanism to skip over layers upon layers of "abstraction", it's better to just not have these layers. There should not be a large call chain that unwraps multiple stack frames in a row. The best code is code that just does what needs to be done in a straightforward way without using any library cruft. Nothing to "optimize" away this way.
Give any concrete example where exceptions are useful in order to work around cruft, and we can see if there is a better way to write it that does not require exceptions.
Frankly it seems hilarious to me that we're having a discussion about optimizing a few if's because they allegedly add too much overhead for the work of skipping over layers of cruft.
In the past years I have seen the need to use longjmp() (which is in a way a mechanism to get exceptions for C) only one single time, when I was coding an interpreter that got embedded into a longer running application. I didn't know how to skip through recursive layers of parsing calls in the recursive descent parser. But I still feel like there is a better way to write the parser (like, pushing work to parse to a data structure instead representing it as a nested stack frame). This could also benefit error reporting for example.
In C, of course, you have no choice; you write the whole function over and over. C programs are, e.g., filled with custom one-off hash tables, because you can't write a performant, general hash table library in C. People using C++ and Rust do not code their own hash tables, because they cannot match the performance of well-tested and tuned library components.
As a consequence, modern C++ and Rust coders get better-than-C performance for a lot less work, and ship with many fewer bugs, by relying on mature, well-tuned libraries. That is worth a lot of what you call cruft.
That said, it is not uncommon to find C++ and Rust coders aping what they see in general-purpose libraries by adding superfluous cruft in their own programs, that provides no such benefit. But that is a complaint for another day.
Taking that back. I forgot for a second that we're focusing on the cases where the exception is not thrown.
> Exceptions cannot be used in code hot paths (~thousands of iterations) when it's not a small perf problem anymore.
You should read the paper in the link. Both of your claims here are exactly backward.
The problem is "just" that exceptions don't scale across multiple threads due to internally using a global lock. They are otherwise much faster than return values (4x faster in the trivial example)
18-16ms (boost::LEAF) vs 19-23ms (exceptions)
https://lordsoftech.com/programming/error-codes-are-far-slow...
> TL;DR On modern 64-bit PC architectures, C++ exceptions only add unreachable code with destructor calls into functions and their effect on performance is below 1%, but such low values are difficult to measure. Handling rare errors with return values requires additional branching that slows down the program in realistic scenarios by about 5% and are also less convenient. If an exception is actually thrown, stack unwinding costs about 2 µs per stack frame.
5% cost to using error codes. In exchange for that 5% you don't fall off a performance cliff if you hit a bunch of errors unexpectedly, and can confidently use them in hot paths and on CPUs with multiple cores.
I'll take the lower perf complexity and greater perf robustness of error codes any day. Reliability and predictability is more important to me than a 5% perf cost.
> TL;DR On modern 64-bit PC architectures, C++ exceptions only add unreachable code with destructor calls into functions and their effect on performance is below 1%, but such low values are difficult to measure. Handling rare errors with return values requires additional branching that slows down the program in realistic scenarios by about 5% and are also less convenient.
Also I see you "cherry picked" your counter example using the sqrt test, which is less influenced by per-call overhead. Using instead the fib test, which is basically just a test of call overhead, we get instead:
12ms-14ms (exceptions) vs. 22-23ms (boost::LEAF)
> 5% cost to using error codes. In exchange for that 5% you don't fall off a performance cliff if you hit a bunch of errors unexpectedly, and can confidently use them in hot paths and on CPUs with multiple cores.
You also get more fragility since error handling is now manual. I'd rather have performance and reliability, which only exceptions can provide. They don't provide that today thanks to the global mutex, but the point of the paper & related explorations is that that both can be fixed, and that fixing it provides a superior performing solution. And a more reliable one.
Unless you're arguing that it's better to just always have a 5% performance tax & manual, error-prone error handling? We shouldn't attempt to fix the toolchain at all? The paper isn't arguing that people should use exceptions today. The paper is arguing that exceptions should be fixed so that they can be used in the future.
Exceptions today are untenable for a library author because you don't know which input data your callers will use, and therefore can't predict how often errors will be thrown.
Ideally we'd fix c++ with better manual propagation of errors and compiler warnings, like Rust or Zig.
Even if the global mutex is fixed, the single threaded perf numbers are terrible if you hit a bunch of unexpected errors. That seems unfixable.
I'd rather have 5% slower code, than 5% faster code with occasional 1000% slow downs (in the worst case).
Perhaps, but that's a strawman through & through. C++ exceptions don't have that performance cliff when thrown, per both the paper in the OP & in your link. They are fairly cheap to throw assuming a single-threaded application.
That is, the fib example with 10% failure rate has the same performance using exceptions & using Boost::LEAF.
Unless you're calling it "unfixable" because there isn't actually anything to fix in the first place...
I disagree with this. I think they should be used whenever "standard" return value error handling starts to obfuscate the normal operation (happy flow) of the code. This impact on developer performance happens a lot sooner than the measurable machine performance.
In rust, propagating an error up is as easy as adding a '?' after the function call.
fn read() -> Result<String, io::Error> {
let f = File::open("hello.txt")?; // propgate up?
let s = f.read_to_string(&mut s)?; // propgate up?
Ok(s)
}Turns out I strongly prefer explicit errors over exceptions, I just also strongly prefer error handling not consuming 75% of the lines on my screen.
And in the cases where I really do prefer a panic, there's always `.unwrap()`
The serialization of stack unwinding issue is a real serious problem.
Well yes, but EH is basically trying to be the best way to do a highly non-local goto with a dynamic target. If that part is your bottleneck than indeed you may have a more specialized requirement.
I do on occasion build a homemade version of a standard library datatype because I have a specialized need and only need implement the subset our code will call. That doesn't invalidate the more general implementations in the standard library (which tend to be quite good, even corner cases).
Likewise sometimes I check for a null pointer being returned or other local error flag. That doesn't mean I don't think exceptions are a good idea.
> The serialization of stack unwinding issue is a real serious problem.
That is the important part of the paper and as I wrote in my comment, I am disappointed that a possible solution, written and tested by the paper's author, was dismissed.
I think it could be done by changing the mangling algorithm (basically: compile fails if new-ABI versions are not available) but I haven't thought enough about it to remove the words "I think" from the beginning of the sentence.
Basically, going forward the compiler would emit two versions of the unwinding tables by default: the legacy one and the one that allows fixing the thread contention issue. The unwinder is changed to use the efficient version by default with the ability to fall back to locking if it encounters a shared object that only has the old version.
Space conscious users could choose to emit only one version by using a compiler flag, and after a decade or so the compiler is switched to only emit the new version by default (but the unwinder could retain the capability to understand the old format for a few decades longer).
It seems like most of the table contents should even be shareable between versions? The locking seems to be only there to guard against shared object loading/unloading which should only affect the topmost level of the data structure.
Admittedly I have no idea how those tables are actually defined today.
Compilers would have to generate both forms forever (except for new architectures) but probably that wouldn’t be terrible.
(I wonder when software engineers will more commonly think about their work on a generational time scale. Like how the cathedral builders of old must have thought about their work...)
In any case this doesn't seem to be an ABI issue
If you write a jpeg-processing service, it’s intuitive to raise an exception on a malformed jpeg, but there’s no guarantee that only 1% of the jpegs users upload to the service are malformed.
In other words, we treat exceptions as exceptions from our code expects, not what’s statistically unlikely in the input space, which is in many cases impossible to predict with accuracy. (E.g., even if only 1% of your inputs are malformed over all time, on some Thursday you may be hit with 80% bad inputs, making the performance drop across the service unacceptable.)
Not really. Having to parse a malformed doc is not an exceptional situation: it's a basic use case, and one which is very close to the happy path.
Which the paper covers, that's the std::expected section: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2022/p254...
As a bonus not only is using exceptions for this faster, it's also less error prone. There's much less risk of any intermediate function or helper abstraction failing to propagate an error code properly.
Well, it would be if exceptions weren't avoided like the plague in C++ because they have a, well, bad implementation, which can't be meaningfully fixed without ABI breakges (as the paper covers)
Which really should be possible with exceptions. It's a shame that it isn't.
So a function that ingests a file and processes it may throw an exception if the file isn't found so that the UI can catch it and ask the user for an alternative filename (or to give up and not open a file at all).
If you're connecting to a remote machine and don't get a response, you might throw an exception because you don't know if the user typed the name wrong.
While if you are already talking to a machine and it stops responding it's reasonable to wait a moment and retry, as if could be a transient network brown-out which is something you can deal with on your own.
C++ stdlib does not throw an exception if a file does not exist. I agree with this philosophy because files not existing isn't an exceptional case.
In a situtaion where are more bad jpegs than good ones, we still don't necessarily care that they cause the slow path, if the purpose of the service is doing meaningful processing with good jpegs.
This is in contrast to say python where the convention is to rely heavily on exceptions as part of the normal flow of the code, sometimes described as "asking forgiveness, not permission". It is not uncommon for a method argument to support multiple types, and to discern them by treating the object like one type and if you get an exception, then try treating it like another type. Likewise, if you're not sure an item is in a dict, you just try to access it, and catch the exception if it isn't. This has performance impacts, but so does everything else about python, so it isn't worth optimizing.
Where I do think this pattern makes sense is in trying to use system resources. Checking that a file exists or a process is alive before deleting or killing it is a recipe for difficult to track down Time of Check to Time of Use bugs. I'm curious if you think this is also an anti-pattern in c++ and if so, how you properly deal with the TOCTU race conditions?
If the library you are using does throw exceptions, then I would probably start with just using exception handling. Then if I notice poor performance, add an initial check, purely as an optimization, while keeping the exception handling to deal with race condition.
So if you are trying to hold a lock you don't throw an exception, you just wait and try again. Perhaps you can't reach that host; try again a few tiles before giving up and throwing an exception. But if you try to write to removable media and the device won't open, all, your program isn't going to mount a tape itself: throw an exception and let the problem be handled at a higher level.
I would have not have thought an unreachable host was exceptional, given that it's quite normal.
The point is that handling the host unreachable is almost always semantically higher level than the code opening the connection. This the code opening doesn’t implement the policy of what to do when the situation occurs. In the case, say, of a hard-coded address then it’s possible the author of that piece of code does know how to handle a host unreachable case.
On the software abstraction, exceptions are simply a convenient implicit control flow device for handling the alternate path when an intended state cannot be achieved. I don't follow the paper's argument that "current exception design is suboptimal for efficient implementation" - but I would not be surprised considering the lifetime issues.
The argument of the paper is really that exception handling shouldn’t use a global lock, a problem made worse by the proliferation of processor cores on modern chips.
Btw, we don't say "exceptions are for infrequent conditions", because that's not what they're for.
I quite like the etymological "taken out", because it carries a notion of special handling - a control flow aspect that is the main point of using them.
Exceptions are exceptional so in principle it doesn't matter (within reason) how long it takes to throw one as long as it costs nothing not to do so. So measuring the cost of repeated throws IMHO doesn't cast light on any useful case, and the approaches that add runtime cost for the path not taken, even Herb Sutter's, are not acceptable.
His code transformation example is simply the compiler behaving properly, unless it can't make that transformation even when foo() is declared noexcept.
The high core count is a real issue and a legitimate reason for an ABI break. Exceptions are the kind of below the surface plumbing that can't reasonably be implemented in regular code.
And in that regard the paper does make a good suggestion, though it then dismisses it! The tree approach described in section 3.4 seems like the right kind of fix.
Ultimately there's a spectrum of branching (`return` -> `break` -> `goto` -> `throw`) all of which need consideration in light of multicore deployment.