You are matching your perception of someone with your pre-existing notions of them - typifying them in a sense. There are so many things to consider, really.
If you see someone in a Lamborghini, you might assume they are rich. But they may in fact be massively in debt. Therefore the analysis becomes a social one - about there access to funds or cars and not necessarily related to their economic status.
If you see a mother with a phone you think she couldn't / shouldn't be able to afford (I realise they used this example as a past belief but may as well use it). The phone could have been a gift, it could have been won - there are so many factors that you are reducing to one.
The point of identifying a trend is to reduce the factors. If there isn't in fact a trend, that's when you look very closely at the individual person, or that's the point where you embarrassedly drop the subject because you made a false correlation.
Side note: I consider you rich if you can make payments on expensive-enough things, even if your debt is greater than your assets.