But doesn't this protocol make it impossible to prove what your secret key is? i.e. there are multiple (user-derivable) keys that look valid, as in deniable encryption?

(And I'm sure everyone's in agreement with the importance of making the ballot secret.)