Note that most keys are level-1 certified, i.e. against online attacks. Physical attacks are generally not much important, because if an attacker has access to your key, he can simply use it. (unless you went through the additional hassle to set a pin, but very few people do it.)