This is a concern that's very often raised. I get the point, but there are two things to keep in mind:

Having aligned interests makes this tractable. When someone raises this concern they typically posit participants who don't have aligned interests.

Secondly, we aren't very good at seeing our day-to-day experience that suggests this is doable. We don't credit our participation in consensus-based groups, nor do we recognize instances of compromising or actions that are not self-interested.