It is extremely rare, nowadays, to "create a destructor" to manage memory. The destructors generated automatically from templates in the Standard Library suffice.
Creating classes is just programming.
So, no, that is not manual memory management.
It's OK that these issues exist - as you say it's just programming - but don't delude yourself into thinking that's automatic memory management. It's implicit, but it's not automatic and that means it's (at least partly) manual. When you say you don't use manual memory management but then say you let standard-library templates handle it, that's a contradiction and tantamount to a lie.
/doubt
Once you're facing, for example, the deallocation of an entire tree that you own and that's going out of scope, your pause is unbounded. Compared to that, a GC'd language could delay the deallocation, or even in case of stuff like C4 have a dedicated thread deal with it.
Sure, you can always write code in a way that avoids this, but non-consing is also an option for many of the GC'd languages (at least the more advanced ones, like Common Lisp).
So, doubt all you like, unavoidable random millisecond pauses are not a problem in non-GC languages, no matter how much you wish otherwise.
It is a fact that the majority of programming in modern non-GC languages, such as C++ and Rust, involves no manual memory management, no GC, and no reference counting (which is also GC). The memory-management automation provided by the core language and the standard library are equal to almost all challenges, all by themselves.
Thus, users of modern non-GC languages do not experience the pauses seen in GC languages, or the unavoidable pointer chasing, or the cache poisoning, or the "reachable leaks" that are GC languages' dirty secret. They are not, in fact, obliged to "trade off" anything at all for being free of those failings.
That is not to say that all design goals are easy to achieve, but overcoming a GC's failings is is not among the activities needed achieve them.