Exactly. You can't have perfect trust in every citizen to keep track of these things. And if it's a government issued ID in the first place, I see no trouble with them having your private key as well. Just don't reuse that key.

This would take a lot of solid investment politically, and technically, to make it even slightly feasible to work for the average Joe.