It can result in a loss of relative status or self-esteem for the people who are fit. If I'm fit thanks to some combination of genetics and lifestyle, this fact is probably a source of my self-esteem, it's something that makes me better than most people in a certain aspect.