This is what governments are for. Sometimes your individual liberties collide with other people's individual liberties, or the collective liberties of the society. You elect a government and then they act on your behalf in order to reconcile these conflicts of interest. The people of the EU elected their representatives. They're free to leave the EU (individually or collectively) if they don't like how that's working for them.

If you're not in the EU and you don't like their terms, don't do business with them.