I don't get it. People form societies because we are better off collectively. We run into problems where some people manage to rig society in a way that some people are consistently worse off and others are consistently better off.

Society as a whole is better off => every single member is better of.

There is no such relationship. However, since every individual directly contributes to the total productivity of society, we want every member to reach their maximum potential and one way to achieve that is to introduce welfare so that people at the bottom don't have to be homeless or starve because we want them to have a second chance at life.