Nothing a society does is in every individual's best interest.

The question tends to end up - are the costs and benefits diffuse or focused. Focused costs tend to be negatives (loud complaints from small groups), and diffuse benefits aren't perceived.

Free trade (at the start) is a diffuse benefit, and a focused injury. It's like arguing for action on climate change.