I see. So the weakest link in the security chain is that someone discovers the <6 digit pin on your usb key (easy on a security camera), and then manages to steal it for 2 min.

Not terrible for anyone whose threat model is not targeted attacks. But quite bad for people whose threat model is exactly that.