I've been writing software for close on 30 years and I know damn well how easy it is to make mistakes. Even on days when I think I'm in the zone, there will be occasions when I'll go back to it and think "WTF was I thinking?" when it bombs out of the first round of automated testing.
Technically, the only dependencies on pointers are types of the allocation/deallocation functions which are mandated in ISO C++ even for freestanding implementations (not the case in ISO C), and their derivations (new-expressions, etc). All other sane uses of pointers should be replaced by some better alternatives when appropriate (shared_ptr, unique_ptr, observer_ptr, reference, reference_wrapper without type completeness requirement, uintptr_t, iterator, etc).
In that sense every pointer is a badly designed variant type; basically each occurrence of a pointer would likely be mostly incorrect (or not correct enough) in program semantics. I always treat that vagueness as a can of worms, so even merely replacing pointer by observer_ptr somewhere makes a big win to me. There do exist real costs like verbosity and bloating of binary size, but they have to be paid for sin of the father. The ship has sailed too far in a regretful direction; but this is not the excuse to fix it - which is one source of the problem from many developers.
You need plenty of tests and valgrind.
Its dangerous for most programmers to be cavalier with owning raw pointers. If we aren't we'll get another Java.
The point of c/c++ in comparision to other languages is that you can do anything and having a huge toolbox for that. Pointers are excelent sledgehammer. You can be perfectly fine with not using them but c++ need to have them. The pointers are not source of the problem, the developer is. That's why languages like java are prospering, it prevent incompetent people making stupid mistakes and that is fine. But some people need and want freedom that c++ offers.