Not necessarily. If this is for a desktop computer you can just leave the USB plugged in and the computer running, as normal. The threat model isn’t too different between that and a normal home/office computer - in both cases you have to trust the physical security of your home/office.

For a laptop, you’re already lugging it around anyway, just remove the USB drive whenever you shut it down.