Additionally, the goal of unsafe is not to throw away all safety; it's to prove to the compiler something that you know, but it does not know. This means that you can contain the unsafety, and still gain the benefits as a user. The goal is to isolate and contain it, making it easier to write correctly.
Others say that they love it because it enforces how they write their code anyway, but now with compiler checks that prevent them from messing up.
I think it's a fair criticism that the borrow checker is sometimes not powerful enough to understand a perfectly valid access pattern, but unsafe is always there if you know better.
More often than not (in my experience), the access pattern is not actually valid though, or risky and easy to mess up. And the compiler rubs your nose in a suboptimal design.
I've found that when I needed to reach for `unsafe`, I often could restructure the code instead to achieve a safe result with minimal performance loss.
Complexity doesn't bother me (obviously, since I use C++ regularly) but lack of expressiveness is a big issue.