> And for the regex example the author doesn't use C++
> std::regex and std::futures (std::async) like he does
> for Rust.
> It worries me though the examples are still using
> threads and mutexes.
> Also lock free data structures dont scale that well
> either. As they still require synchronization and that
> hurts scaling
That was the dream of the mid-2000s. But now hundreds of cores are never going to happen. The CPU vendors have decided that the industry has run out of time to parallelize their programs and are now refusing to scale up. Our hope for speedups now lies in using SIMD and GPUs effectively ("heterogeneous computing").
I'm not sure about the details for the "fasta" benchmark, but I know it's at least partly I/O bound, and many implementations for the benchmark are single-threaded:
https://benchmarksgame.alioth.debian.org/u64q/performance.ph...
Notably, Rust also edges out a multi-threaded C implementation for that one. I'm sure they've done some very hairy optimizations to get to the top on that board, but it's cool to see that it's possible.
On the subject of regexes, I am not familiar with the C++ std::regex implementation, but it does look like (some rather slow) C++ implementations are using a boost regex library. Those are generally fairly "standard" in C++, right?
The most important optimization I did was to avoid regex machinery as much as possible. In particular, Rust's regex library has very good support for prefix literal optimizations. In particular, regexes like `abc[x-z]foo|bar` will compile down to an aho-corasick[1] automaton over the strings `abcxfoo`, `abcyfoo`, `abczfoo` and `bar` with the failure transitions completely evaluated. (The end result is an automaton represented by a matrix. This is memory intensive, so only prefix literals of a certain size can be accommodated. But you don't need a lot to realize huge gains!)
I wrote more about it here: https://www.reddit.com/r/rust/comments/39unje/ahocorasick_fa...
Hint: most of the benchmark game regexes are relatively simple and compile down to either simple `memchr` calls (most regex engines will do that, nothing special) or a aho-corasick DFA that completely avoids the traditional regex evaluation machinery.
In general, regex implementations differ dramatically in the optimizations they perform. It's hard to draw conclusions about them without some explicit knowledge of how it is implemented.
About the regex example.
https://benchmarksgame.alioth.debian.org/u64q/program.php?te...
The boost regex library is fast and widely used. It doesnt look like the example is using it though, but a library called "re2". Never heard of it before. Googling turned up this https://github.com/google/re2
https://benchmarksgame.alioth.debian.org/u64q/program.php?te...
https://benchmarksgame.alioth.debian.org/u64q/program.php?te...
I have no idea whether it's the regex implementations that are causing the comparative slowness, to be honest. Just seemed interesting that some C++ implementations using what I would think are common techniques fall behind many other languages' implementations.
Of course, benchmarks are always arbitrary, and some of them will just disadvantage approaches that would be perfectly realistic ways to solve "real-world" problems, so it's always a grain-of-salt situation. However I'm not sure it's necessarily fair to say that "C++ should be at least as fast as C for any fair benchmark." Doesn't that expose you to the danger of redefining fair benchmarks as "those benchmarks which reinforce my preconceived notions of which tools have what performance characteristics"?
On the benchmark though, its doesn't seem quite fair. At least that first one is comparing a single threaded c++ program, to a multi threaded rust program. And for the regex example the author doesn't use C++ std::regex and std::futures (std::async) like he does for Rust. It makes me think a proper C++ implementation would do much better. (Personally I use the Parallel Patterns Library. I like it a lot. We will get something similar in C++ in a few years)
I don't know much about rust. It worries me though the examples are still using threads and mutexes. I think we need much higher level abstraction (like ppl::task, coroutines, etc) to get better scaling. Also lock free data structures dont scale that well either. As they still require synchronization and that hurts scaling). I think we need some kind of revolution in how we code to scale our programs to hundreds of cores.