Most people want to live in a society that maximizes positive freedom, or some balance of freedom and prosperity. In those societies, it is considered legitimate for the government to use force to stop people whose actions prevent members of society from having positive freedom/prosperity.

Of course that is a very simplified description. In practice, most societies promote a balance between positive and negative freedom, recognize some limits on the government's ability to use force, recognize some degree to which people can choose to act in ways that don't promote positive freedom/prosperity, etc.