> there's no legitimate way to decide where the exact appropriate middle ground is. That's something we all have to decide for ourselves, and it can't be forced upon others.

This statement is false. Governments and judicial systems around the world routinely decide where the exact appropriate middle ground is, and make binding judgements forced upon people by threat of fines and imprisonment, ultimately acts of violence.

Some of it is even democratic (deciding together, not each for ourselves).