You're right, it's not. I don't really care about open source. I don't trust Google and I don't like their intrusive tactics. I'd actually feel more comfortable with Microsoft, as they probably have more layers of management preventing stuff getting done. I can't believe the part of the company that mismanaged the largest IM network into nothingness is competent enough to really invade my privacy to a disturbing degree.
More what I care about is making sure I can run software I'm interested in, and have my device do what I want it to do. Tethering (on a phone) regardless of carrier policy, for instance. Apps that might violate store policy -- I simply don't have the potential worry with Android.
I don't want "open" necessarily, I want "respect" (for my privacy).
Unless you're going to inspect all the code/apps running on your system, an open-source OS isn't necessarily going to guarantee your privacy.