Memory management in modern C++ is considerably easier than it used to be. Bespoke memory managers aren't really needed, you can do almost anything you need to without ever using new or delete.

Out of curiosity, what is the field for which you find this to be true?

If you are writing C++, any field! Just use std::shared_ptr and std::unique_ptr from the standard library, along with std::make_shared and std::make_unique.

C++ shared_ptr unfortunately is artificially slowed down by multithreaded synchronizations. The count in the control block is always updated atomically. But I frequently need to use them not because my data is shared by multiple threads, but because the ownership situation isn't static. Consider for example implementing a single-threaded persistent tree.

This means people frequently need to reinvent their own reference counting mechanisms.

Rust does this right. Rc and Arc.

To be fair, std::shared_ptr is just a high-level component that's a part of the STL. Although it's defined in the C++ standard, it's just a generic shared pointer implementation designed to be as robust and bullet-proof as possible.

As with everything in the C++ STL, if you care about bleeding edge performance then you should be prepared to use more perform any (and less generic) components, whether it's data structure implementations or shared pointers.

This is the quickest way to kill your performance in C++. I guarantee it wasn’t what Carmack was talking about.

I used sharedptr extensively in a game engine. Whoops: suddenly 20% of the frame time was gone, never to be recovered. Once that performance is gone it’s almost impossible to get it back, short of rewriting every system.

Only use shared_ptr when you actually want shared ownership. If you stick to unique_ptr for owning references and then "borrow" that raw pointer in functions, then you get speed without sacrificing too much safety and still no new/delete.

Additionally, shared ownership should be rare. Most objects should have a single owner that is responsible for them. This is not only better for performance, but also makes the system easier to understand.

Yes shared_ptr is relatively expensive; avoid it in your inner loops, but know that it's there for the other 80% of your code.

On the bright side, you didn't lose 200% of the frame time!

And, use unique pointers where possible. I see a lot of code using shared ptr where unique would suffice, even in my own code.

std::shared_ptr uses CAS atomics, which in heavy multithreaded code (multiple threads operating on the same pointers), can have surprising overhead in some situations.

On the other hand, not using atomics in heavy multithreaded code is just a recipe for disaster. The problem with shared_ptr is the fact that it uses atomics even in single-threaded code, which is obviously an overkill.

> The problem with shared_ptr is the fact that it uses atomics even in single-threaded code, which is obviously an overkill.

On the other hand imagine the security issues if shared_ptr was not thread-safe - you could just not reliably destroy a shared_ptr in any thread.

If you really know that you're going to get a graph of shared_ptr that do not move of a single thread, you can use boost::local_shared_ptr explicitely.

One of the advantages of rust is its ability to segregate safe-to-share (between concurrent contexts) and not so, at compile time.

While it can’t (yet?) be generic over them, it lets you use the non-atomic Rc in thread-bound structures and know this is never going to be shared between threads, whereas the more expensive Arc can have handles be moved from one thread to an other.

Counterparts for Rc [1] and Arc [2] (and compiler enforcement of their safe use [3]) are available in C++, though not standard.

[1] https://github.com/duneroadrunner/SaferCPlusPlus#reference-c...

[2] https://github.com/duneroadrunner/SaferCPlusPlus#tasyncshare...

[3] https://github.com/duneroadrunner/scpptool

> Memory management in modern C++ is considerably easier than it used to be

It was never actually an issue

... until you got the segmentation fault, that is.

If you're parsing market data then you shouldn't really be allocating at all once the initial setup is complete. So there shouldn't be a need for ref counts.

This is because the allocator itself can have an unbounded runtime that takes milliseconds, causing you to drop.

In the past I've replaced malloc with an implementation that asserts if called on certain threads after init time.

Milliseconds??! What causes that to occur? Though I suppose if they can take that long, avoiding them is an even better idea than it is normally...

For constant time allocations, try TLSF: http://www.gii.upv.es/tlsf/

Most likely it is simply invoking a syscall (sbrk or mmap), which invokes the scheduler and may yield the time slice. Feed parsing code may not issue any other syscalls (entirely userspace), turning the allocation into a bit of russian roulette.

I'm not the op, but I'm guessing large amounts of fragmented memory is what causes that.

It's one benefit of a gced language with compaction, allocations are typically bounded (except when they trigger a gc).

Except compaction can also take many milliseconds and come from different threads.

Writing a trading system in Java is harder than C++ imo because where before you had an allocation problem, now you have a multithreaded randomly stalling allocation problem.

Virtu did it but everything I’ve heard about it nullifies the benefits of using java in the first place.

Newer Java GCs are very low latency (microsecond). You trade performance and memory for that low latency though. AFAIK, they are still compacting.

Still though, probably makes sense to do it in a lower level language. It's just far easier in C++ to decide that "Hey, you know what, I just want a big memory block that I control".

I've even heard of game devs doing things like having per frame allocators. They get super fast allocation because they just pointer bump and at the end they simply reset the pointer back to location 0. I'm sure trading systems could do something similar.

The point is that a one millisecond pause is unacceptable. Low latency Java GCs have average latencies of one millisecond, 99th percentile latencies of 10 milliseconds, and 99.9th percentile latencies are neither measured nor optimized for.

I don't consider it realistic to think that garbage collected languages might ever be usable in the context of game engines or HFT.

> game engines

Game engines are already written in GCed languages. Java in particular.

You may be right that a 10ms pause is unacceptable for HFT. However, for a FPS, 10ms is more than acceptable. It translates to 1 or 2 missed frames in the worst case.

A bigger issue with using Java in particular for games is it's lack of value types. Writing high performance code for java is just that much harder because the language gets in the way.

The only serious engine I know of that's written in Java is Minecraft and its performance problems are notorious.

Microsoft XNA and Unity's .net support were also pretty popular but the popular games written in those languages (like Bastion) didn't have many heavyweight assets or allocator pressure.

I've seen state-of-the-art low-latency trading software written in C# or Java.

You should not be doing any memory allocation in the critical path anyway, so garbage collection is hardly a relevant factor.

Having written lot of Java, Scala and C++ (and recently some Rust), I must say it is much easier to avoid heap allocations in C++ and Rust than in GCed languages, thanks to explicit allocation on the stack and pass by value + move semantics.

A big push in .net core 2.x and 3.x was what we call the Span-ification of the base class library and the runtime. This means there are many new APIs for dealing with slices of memory in a non-allocating manner, and this combined with memory pooling has contributed greatly to an overall performance boost to the runtime by reducing copying and GC time. These same APIs are available to the developer so I'd imagine that it would be simple to build a non allocating network buffer reader. I have built a non-allocating video renderer before using the new APIs, for example.

There exist libraries for native memory pooling in Java as well, and we're using them. I'm not saying low allocation code can't be done in C# or Java. But these languages don't give some nice tools that are present in C++ and Rust - in particular RAII and automatic reference counting.

> I'm not saying low allocation code can't be done in C# or Java.

With modern tools and APIs it should even be possible to write alloc-free work loops in C# (though you're probably writing pretty alien C# at this point). AFAIK it'll remain impossible in Java until the "value types" effort bears fruits.

A few months back a study of sort made the rounds implementing a network driver in multiple languages (Ixy), C# did extremely well in it (better throughput than Go and near competitive with C and Rust at higher batch size though it was way behind on latency) while Java was pretty much in the dumps.

In one of the reaction threads (don't remember if it was on HN or Reddit) one of the people involved explained the discrepancy between Java and C# by not being able to go under ~20 bytes of allocation per packet forwarded in Java.

There were other odd / interesting results from the effort e.g. Rust was slightly slower than C, in investigating that they found out Rust executed way more instructions (especially significantly more stores) but had significantly higher IPC and much higher cache hit rates.

ixy also tried just converting the C code to Rust (using C2Rust) then compiling it and that turned out to be very slightly faster than the C code, which was funny: https://github.com/ixy-languages/ixy-languages/blob/master/R...

A trading system in C# or Java is not, practically by definition, state-of-the-art. To suggest otherwise just demonstrates not knowing the State of the Art.

"Capable" is plausible.

Sorry, I doesn't make the rules.

But aren't the edge cases the things that satisfy your pay? The standard cases of today are the introductory examples of tomorrow and will be automated or abstracted away by the end of next week. In JavaScript.