Note that hardware wallet attacks tend to get published at least once per year, but so far most/all of them have relied on physical access. Not just sending some buffer overflow via the USB link, but actually opening up the device and messing with capacitors or something.