Who delivers those letters? In the U.S., and many other countries, it's the government.
You're literally handing your letters to the government and asking them to deliver them to the intended recipient without peeking. In short, you are trusting them to do the right thing, because they certainly could look inside the envelope if the really wanted to.
Legal safeguards are your only defense against the government here, but we the people generally consider it a solid safeguard.
But if it were to even be suggested to send email using government-provided networks (even encrypted email) you'd be laughed right out of any hacker con you attend.
It's a pretty surreal difference IMHO.
If I was a person of intense interest to the government I definitely would not trust the mail system. But as a nobody, I imagine its pretty unlikely anyone is looking at my mail.
With email, the resources required to record all email is comparatively trivial and this makes it much more feasible that even us nobodies are having our privacy invaded.
But for the vast, vast majority of people it's apparently not something they worry about. They indeed simply trust the government to obey the law.
Luckily, the vast majority of people aren't really needed to bring about change because most people do nothing because their blind trust doesn't require them to… Only just a relatively few methodically dedicated people who have the ability to see beyond and work through their current circumstances and help forge the future they want to see, are all that are needed.
What if everyone believed that law-abiding citizens should use postcards for their mail? If some brave soul tried to assert his privacy by using an envelope for his mail, it would draw suspicion. Perhaps the authorities would open his mail to see what he's hiding. Fortunately, we don't live in that kind of world, because everyone protects most of their mail with envelopes. So no one draws suspicion by asserting their privacy with an envelope. There's safety in numbers. Analogously, it would be nice if everyone routinely used encryption for all their E-mail, innocent or not, so that no one drew suspicion by asserting their E-mail privacy with encryption. Think of it as a form of solidarity."
Phil Zimmerman, 1994
http://www.pgpi.org/doc/whypgp/en/