Have a vector of shared_ptr that own the objects in the graph and build a graph with weak_ptr ?
Removing an object is just as easy as removing an element from the vector. (If you test the weak_ptrs on use, that's actually the only thing you would need to do).
shared_ptr means reference counting, reference counting means you lose determinism because you no longer know if releasing a reference will trigger releasing a resource. Delay and offload releasing the resource to a separate thread, you lose your guarantees when a resource is actually freed, too, just like using a garbage collector.
And I know it for the .NET GC, they tried a reference counting GC as alternative to a collecting GC and it performed worse and comes with the cycle trouble.
Isn't that equivalent to using a shared_ptr directly, just unnecessarily complicated?
Reference counting works fine as long as you don't have cycles, of course.
The solution above works even if your graph has cycles. Of course if you know that it doesn't you can just build the graph with unique_ptrs.
It works with cycles, as long as you can know exactly when you want to remove something from the graph (as opposed to having it be removed when no longer referenced). Reference counting works fine that way too, though. It's a bit more work, but you just dive into the structure and manually remove references which breaks any cycles it may be involved in.
Imagine you have an arbitrary long lived connected cyclic graph that can be incrementally updated from multiple short lived worker threads. With a GC, this is a no brainer: just put the objects in the graph. No workarounds, no extra tracking.
Without a GC, on removing a node, you have to walk to essentially do a mark/sweep of the graph to find dead nodes that were connected through the node you removed.