There is an additional key/identifier that you don’t have to carry with you that would prevent them from logging in even if you had to give them your vault password.