When did privilege become bad? I mean it is a reality we face daily but is it actually "bad" so should be eliminated? Plus there is no one privilege right?

At what point do I owe someone something for being less privileged than me?