A long time ago, in a 'Modesty Blaise' book, when Wille Garvin is incarcerated and the guard tries to cut a deal with him saying that he'll set him free - Garvin says "I already have my freedom, what you'll give me is liberty".

I always think of this phrase whenever questions of freedom come up.

And like you mention, all of us our free, but mostly our liberty is curtailed by external circumstances or by our own fears of retaliation.