You could publish hash(salt + SSN) = Vote for all votes on a website, and therefore if somebody voted twice they'd hit the same hash twice.
When you vote, you'd get a physical ballot, and a physical receipt, digitally signed by the booth. If what is published online Hash(salt + SSN) doesn't match your receipt, you'd have physical signed proof of tampering.
If you don't have that list, then there's nothing preventing you from padding the votes with fake people. You can try and implement some way of tracing the votes back to people but, first of all see point 1 up there (oh your phone number/email address/home address changed/was misspelled/you're homeless/you're a nomad) and second, you very likely break secret voting.
FWIW there is no perfect voting system--that's the core problem w/ this discussion. We have to choose:
- Do we have a citizen ID (there's a whole ball of wax here, primarily what happens if someone steals or you forget/lose your ID)
- Do we think some people w/ a citizen ID still shouldn't vote (17 year olds, the incarcerated, etc.)
- Do we create a potential link between people and their votes
- Do we allow people to not vote
- Do we require people to register before voting
- Do we allow people to verify their vote was counted
- Do we allow people to verify their vote was counted correctly (unclear how this would work)
- Do we allow people to vote remotely
- Do we create a paper trail
- Do we allow people to fix problems with their ballots
- Do we require people to sign their ballots, and do we require signature verification
Many of these things are at odds with each other, and depending on how you flip the bits, different political interests will win out.
That's the second reason this is such a nerd snipe. It really, really feels like this is a technical problem but it isn't. It's political, cultural, and societal. You're not gonna fix this with an app or an algorithm. If/When we do fix this I'm sure they'll be involved but, the reason will be multipartisan, multicultural consensus that we need to move forward.
It also possible for an E2E system to have design flaws. For example Scytl created an E2E system for the Swiss that had a cryptographic flaw that would allow a mix operator to modify votes in an election undetected.