This quickly goes down the rabbit hole of identity and identity theft in the USA in a more broad sense. Our entire system is build on similarly flimsy "identity" from banking, to medical records, and even getting copies of a birth certificate/driver's license.

I would say though that "scale matters." If an electronic voting system is compromised the "scale" could be the entire state, and millions of votes, but each time a mail ballot is stolen via identity theft, you're risking it being detected/getting caught, so it ultimately scales to fewer than thousands of potentially problematic votes (which is bad, but not democracy breakingly so, like electronic/computer voting insecurity).