Passports are optional while national ID card would be required and likely issued at birth. People tend to have an aversion to things that are required rather than recommended. Also, while passports are in theory only used when travelling across borders, ID cards would almost certainly be used domestically.

Personally I don't think ID cards are a huge threat to civil liberties - here in America Social Security cards basically already serve the same purpose, for example - but I can see how easy it is to draw that conclusion about a mandatory program to give every citizen something the government can use to identify them.