The point of "because you're white" is, you have no personal experience in that matter. Despite an obvious lack of expertise, privileged people (and I consider me part of that group) feel that their opinion is of at least equal value, and that it should be listened to.

Personally, I quite easily give my opinion unasked (like here), and do feel rejected if it isn't listen too. Likely, because I grew up the way, that my opinion matters.

I would imagine, the experience would be quite a different one, if your thoughtful opinion would be voiced after being asked for it.