As someone who was homeless as a teenager, it would never bother me that a co-worker grew up rich.

Now, I might attribute some of their attitudes to how they grew up, but it wouldn't affect my ability to deal with them on a day to day basis.