I'd have to disagree in this specific context of high-performance parallel numeric simulation. It is terribly easy for memory management errors to produce incorrect simulations that still "work" perfectly, because incorrect handling of floating point numbers will never crash the program, merely corrupt the results in arbitrary, but not necessarily easy-to-detect, ways. Bugs that cause visibly obvious incorrect answers or crash the program are far preferable to bugs that cause subtle errors. It's too easy to "correct" for a bug in a simulation by simply poking it until the buggy simulation seems to return to the behavior you expect, but two bugs don't make correct code.
If you don't really care about accuracy, it may not be a big deal, though... the "shape" of these errors is very difficult to wrap one's head around and I'd still question someone working for or with me fairly hard even so if they think they don't care about accuracy and so it's OK to write code that may or may not be thread-safe.
Something glaring would be if we were getting values that were zeroed out or something like that, but when the only difference is that a slope is -1.4 instead of -1.6 or something like that, you don't immediately realize something's up.
Once we realized there was a bug, it took a long time to track down, and after that I was so frustrated I started writing a new library from scratch in Rust for our research and it worked out pretty well.
Anecdotal evidence for sure but it was really nice not having to triple check for memory race issues.
Or did it not happen in that case?
I'm not even gonna comment on the rewrite, that reads like an ad. :-)
Many simulation algorithms involve manipulating large arrays of data. Rust can check that your indexes are within bounds, but it can't check that your indexes are correct. If you access X+1 to get the element to the left of X, but you should have accessed X-1, then you have introduced the same sort of error as you'd get from memory corruption (and, frankly, one that is much harder to debug. Standard library assertions, Valgrind and ASan are no help at all if you're accessing valid data.)
Nevertheless, I found it helped a lot to compile with `-D_FORTIFY_SOURCE=2 -D_GLIBCXX_ASSERTIONS -D_GLIBCXX_DEBUG` during my normal development. Among other things, that adds bounds checks to all container accesses. I leave them off in release mode, as my full-scale simulations already take multiple hours and I need the performance.
I don't think OP said anything about writing code that's not thread-safe. They specifically said they're carefully designing such code. What they said is that due to the way they handle memory, most related errors don't happen in practice - which is perfectly believable. Floating point errors are separate issue, they could happen in almost any language.
This is a personal quirk, but I consider C++ a different language from C++ + a lot of static analysis tools, on the grounds that most of my major beefs with C++ (and C) can largely be addressed by using powerful static analysis tools from day one. So from my personal, idiosyncratic perspective, I see two very different situations if the original poster is "using C++" or "using C++ and a suite of quality static analysis tools".
I prefer languages like Rust that take advantage of our decades of experience with C/C++ and write the most important stuff directly into the language, but in 2019 I still concede that "C(++) + good static analysis" is still a very economically compelling language for a lot of companies with existing C/C++ codebases. "Upgrading" to C+++SA is not cheap, but it's cheaper than any of the other options, as much fun as "rewrite in Rust" may be.
As for their belief that their code is thread safe, if they are using C++ and not C+++SA, I echo mike_hock's comment. It may be threadsafe, but from this distance and without a lot more evidence, my Bayesian priors say I don't believe it.
I guess I'm somewhere in between on that spectrum. I use clang-tidy and valgrind constantly, along with an everchanging set of profiling tools that occasionally find a potential bug as well. I've used cppcheck in the past, but didn't find it particularly useful, and I use whatever VS2019's static analysis tool is called now if I'm on Windows. I'm also extremely pedantic about compiler warnings, in that I either have a specific reason for disabling a warning entirely for that codebase or it will get fixed. Exhaustive (often mathematically exhaustive) tests go on top of this.
That being said, I don't use any of the sanitizers in my usual workflow. The reason is simple - most of them don't offer anything I need. The huge set of bugs around excessive dynamic memory allocation simply doesn't arise when you don't do dynamic memory allocation, so ASan/LSan/MSan don't give me much. UBSan might theoretically help, but only with issues that have so far been very trivial to debug (accidentally misaligned pointer in explicit SIMD code and the like, results in instant crash and an immediate fix). The one exception is the ThreadSanitizer and when I happen to finally get a non-trivial deadlock, that'll certainly be my first stop. As it is, most of the multithreading I've done recently has been fairly trivial from a compsci point of view (and for good reason, overly fancy algorithms aren't often particularly performant on current architectures).
Regardless, I'm entirely open to the possibility that I've neglected a tool that might help me further. Strong tooling is one of the major advantages of C++ and I've tried most of the major tools at one point or another. I do require demonstrable benefit in the context I work in to continue using them, however, and I've often found it worthwhile to simplify my workflow by cutting something out that turned out to be mostly superfluous.
Use-after-free, concurrent read/writes, forgetting to acquire a lock are all problems that can crash an application or cause correctness issues. Preventing all of those falls under the "safety" heading, and is still useful in a security-permissive environment.
Now, what do you mean by "investing so much effort"? At the language level, it's valuable effort that lets us achieve security in many security-conscious applications, and we get the safety for free in less security-critical environments as a knock-on. Score! From an application developer's point of view, though, you're right. It might genuinely not matter, especially if you're not otherwise motivated to learn the language (a lot of that effort is a one-off and is amortised over your lifetime as a developer, if you choose to learn it). Not all applications benefit that much from the safety guarantees, and you might just be better off not even putting in the effort of using the language if something else is a better fit for you.
If the security requirements are permissive, what's the point of investing so much effort in preventing only one category of bug? There's no misconception.