"Fine" is probably pushing it, but "not the biggest problem you are likely to have" is probably true. But, like, it's easy to accidentally get yourself into situations where somebody forgot to use std::weak_ptr somewhere and you start leaking memory.

Unless you know you must share objects with multiple potential owners, though, std::unique_ptr is a lot wiser.