The physical analogy falls apart because generally envelopes are tamper-evident against most attackers (i.e. it's hard for a malicious postal worker to take your envelope, open it, read what's there, then repackage it without it being detectable that something fishy happened). With communication protocols over the Internet, the tamper-evidence falls apart. All packets look alike.

Another difference: The Internet as a whole routes packets mostly concerned with reliability, not trustworthiness of the ISPs in the middle, so a trusted Postal Service doesn't work on the Internet either.