The library should live in its own sandbox, separated from the rest of the system including any other libraries. All system calls have to go through a permissions system that (e.g.) denies access by default.

There is at least one language that tries to do that (someone posted it here in a comment some time ago, but I can't find it).