Passports cost money and are only issued to citizens. An ID card could be issued to permanent residents, long term visa holders even. It would be smaller in size, have a public key based digital signature chip on it, etc. Basically more of a non-crappy social security card.