No, that's how the feature is supposed to work.

You design an abstraction which is unsafe inside, and exposes a safe API to users. That is really how unsafe it meant to be used.

Of course the standard library uses unsafe. This is where you want unsafe to be, not in random user code. That's what it was made for.