On Android, this is not the case; if you use the Facebook or LinkedIn apps, you will be forced to give up access to your contacts (unless you root your device). The reasons for this are twofold. First, Android permissions are all-or-nothing, which means that apps have to over-approximate their required permissions. Second, because granting access is so easy (no UX hurdle to speak of), there's really no downside to doing so (which is to say, conversion rates will not decrease substantially).
The issue of how the device gets access to the data (e.g. whether contacts are cached locally or are purely remote) is really not the issue. You could easily implement the Android API with no local caching of data, and aside from the performance hit the behavior would be identical.
I could certainly imagine a version of ChromeOS which supports Android apps without the Android permission model, essentially by returning empty data sets whenever the app asks. This would be compatible with ChromeOS's current security policies, and would protect user privacy. But it would be so easy for Google to break that line in order to promote "better UX" for apps in the style of Android, that I can't honestly convince myself that they'll never go down that road. And that's why I can't be entirely comfortable with believing the ChromeOS security model will remain intact.