You only have to update the counter when creating (or destroying) new "owning" references that might outlive any other reference to the object. This is a rare operation, and doing it potentially from multiple threads in a way that requires atomic update is even rarer.
> Additionally, if there is a big data structure in use, reaching zero will trigger a cascade deletion events reaching stop-the-world like effect.
This is a side effect of deterministic disposal, of course. And it only happens when freeing up large object subgraphs, not just any and all "large data structures".
Yes, C++ implements reference counting in a non-ideal way. Swift is also quite bad. But one can do better.
Even a deletion cascade won't stop the world: only the single thread which reached zero will be "stopped". While in tracing GC, a "stop-the-world" stops all threads, even the ones which aren't doing any allocation or deallocation at the moment.
If you add multi-threading and multi-core, there is lock contention for every single memory access to update the related counter.
Additionally, if there is a big data structure in use, reaching zero will trigger a cascade deletion events reaching stop-the-world like effect.
Which, depending on how the deletion was implemented, might trigger a stack overflow in very deep structures.
Herb Sutter has a very interesting talk about the bad issues of reference counting and how to work around them.
CppCon 2016: Herb Sutter “Leak-Freedom in C++... By Default.”
https://www.youtube.com/watch?v=JfmTagWcqoE
There are multiple ways to optimize reference counting, like deferred counting, cycle collectors and a couple of others, which when you are done with all them, ends up being quite similar to having a tracing GC anyway.