The Cost of Exception Handling

Re:The Cost of Exception Handling by mxmlnkn on Saturday November 12, 2022 @08:59AM (#33573090)

It is interesting to read how the compiler behaves when using instructions but the number of generated instructions is only one metric. As always, there is nothing better than measuring the runtime of your actual implementation.

I'm writing this because I had a tight-running loop that returned a bool and in 1 out of 100k or so cases, it would return false. It turns out that instead of checking that bool in each loop iteration, it was actually significantly faster to check nothing and throw an exception in that very rare case. Something like this:

  try {
      while ( true ) {
          foo.read();
      }
  } catch ( const std::exception& ) {
      // foo.read throws on eof
  }

instead of this:

  while ( !foo.eof() ) {
      foo.read();
  }

This might not be the most C++-like code but it saved me like 50% of runtime in one case. Afaik as long as exceptions are not thrown, there is zero overhead, which would explain this. And, the call to eof might have been too complex and long-running, maybe I could have optimized that one instead. But there are abstractions that might hinder such optimizations.

[ Parent | on HN ]

Re:The Cost of Exception Handling by throwaway9870 on Saturday November 12, 2022 @09:17AM (#33573243)

You are making two functions calls in the second loop, vs one in the first. If the result of read() could be checked for eof, then you would remove one call. Also, since the branch is so easily predicted, I would be surprised if there is much of a performance hit once the second call is removed.

If you see a 50% speed-up in the above code, I question what read is doing and the implementation of eof(). I would expect read() to dominate the if statement unless eof is extremely expensive. I have written dozens of read()/eof() functions over the years and read() always dominates because the read involves copying memory and/or making OS calls, etc., while eof() is a simple comparison. So if read() dominates runtime, example 1 and 2 above should not be anywhere close to 50% different.

[ Parent | on HN ]

Re:The Cost of Exception Handling by mxmlnkn on Saturday November 12, 2022 @10:48AM (#33574138)

That particular code came from a bit reader class. So, if you are reading only a few bits per call, then the read call becomes very cheap. It might be as "simple" as this pseudocode

  if ( offset + requestedBitCount < 64 ) {
      offset += requestedBitCount;
      return ( bitBuffer >> offset ) & nLowestBitsSet( requestedBitCount );
  }

And yes, I think the eof call might have been too complex especially in contrast to the short read call. It probably should only check a flag that might be set by the read call automatically.

[ Parent | on HN ]

Re:The Cost of Exception Handling by throwaway9870 on Saturday November 12, 2022 @05:24PM (#33577771)

I still don't get it. You need to put an if statement in that function to raise the exception. So I still don't see where 50% comes from.

[ Parent | on HN ]

Re:The Cost of Exception Handling by mxmlnkn on Sunday November 13, 2022 @03:03AM (#33581020)

At the very least you are redundantly executing logic without the exception. The check for eof has to be done implicitly anyway inside read because it has to fill the bit buffer with data from the byte buffer or the byte buffer with data from the file. And if both fail, then we already know the result of eof, so no need to duplicate checking for eof in the outer read calling loop.

Here is the full commit with ad-hoc benchmark results in the commit message:

https://github.com/mxmlnkn/pragzip/commit/0b1af498377838c30f...

and here the benchmarks I ran at that time:

https://github.com/mxmlnkn/pragzip/blob/0b1af498377838c30fea...

In this commit, I even replace std::optional with an exception. But, I didn't document benchmark results and the commit message begins with "try" so they probably weren't significantly better but also not worse or else I wouldn't commit it:

https://github.com/mxmlnkn/pragzip/commit/0f1babaf3b9cbe2912...

At the end of that file are infrequently updated results of the benchmarks.

As you can see, it's part of my random-seekable multi-threaded gzip and bzip2 parallel decompression libraries.

[ Parent | on HN ]

Re:The Cost of Exception Handling by jnordwick on Sunday November 13, 2022 @07:55AM (#33582610)

Wihtout seeing the assembly it is difficult to say but for such a large difference, the culprit often seems to the loop iop cache. If you check the alignment of the top of the loop instructions (and it inlined away the calls) sometimes you can see an alignment change for some trivial bullshit that has vasat performance impacts because it causes a series of other missed optmizations.

That damn loop cache is great for performance when you hit it,but sucks horrible when you don't even think out it and spend days trying to figure out some performance quandry.

[ Parent | on HN ]

Re:The Cost of Exception Handling by rightbyte on Saturday November 12, 2022 @11:28AM (#33574548)

In the exception code a foo.eof() check need to be in foo.read().

So I guess the runtime in theory should be the same and the implementation happened to be slow. Maybe eol() did some syscall each time instead of caching a EOF int like in C.

[ Parent | on HN ]