Indeed. I hold a similar dual position and it is very easy to rationalise. Government controls the police and military, and so has the capability to threaten me in a way that pretty much nothing else does. I want the process of controlling the government to be transparent, difficult and controlled - hence voters should be identified on voting. I want the government's power to be controlled - hence I don't want them to have a database where they can quickly and easily match bits of my data up.

There tension between those two positions is of little practical importance. I'm not worried that they can link my name and address, I'm worried about a sudden outbreak of majority-vs-miniority where someone I really don't like gets in to power and links, I dunno, tax records-name-address, racial-history-name-address , travel-history-bank-account-photo or somesuch. I don't think it is practically feasible to target someone off the information in a voter role.

The real issue to me is that there isn't much organisation against governments laying the groundwork, and in practice opposing an ID card doesn't mean anything. In Australia we basically have all the infrastructure for a national ID card except the formalisation of a little piece of plastic.