No, it's not a security feature. If you do allow contact access, but don't add people to your address book, you see the name they chose for themselves. And it looks different (slightly faded gray instead of color coded, a tilde in the beginning, slightly smaller font). At least on my Android.

That's exactly what I was referring to. I think I have misunderstood the parent comment. I thought he was complaining about this being the default behavior when you don't enable access to your contacts.