You misunderstood. Since your keys are stored encrypted in a safety deposit box, it doesn't matter if you're forced to tell your password to some random thug - they don't have access to your keys.

You can also put your password in your will, if you're afraid of not passing it to your family.