Well absolutely there has to be an alternative way in, be it serial console or a dedicated admin user who does not have keys in ldap. With sshd you would specify a local key instead of ldap for that specific user, for example