Also freedom of association is very clearly intended as freedom to associate, not freedom not to associate, so it's a little confusing for you to refer to it.
If I wanted to go to a coffee shop and that shop wanted to serve me coffee but the mafia is standing outside saying I have to pay them before I can go in, they are in fact forcing me to pay or turn away.
[edit] I’m getting the sense that if we just restructure this so the union exists and does normal union stuff but you don’t have to “be a member” (but do have to abide by its decisions or else leave, as with a board) you’d be ok with it? Like it’s mainly a semantic problem?
This model does not put you in a group anymore than being an American citizen puts you in a group by giving you the right to vote on things. In my version it is like how you get to be in a political party if you want to and not be in one if you don't, but you get to vote either way. The right to have some say against your manager's power in your life (short of quitting, that is) should be unassailable.
The fact that quitting is "all or nothing", and often not an option to people because the cost of finding a new job is high and often prohibitive to specifically the people who are most disenfranchised, makes it a poor and almost useless way to exercise power. Unions allow, in principal, for there to be intermediate pressures between "quitting" and "asking nicely".
Not all unions are great, but the median unionized worker earns more than a non-unionized worker in the same field. So the main reason people wouldn't want to join a union is because they've been conditioned to think they're bad, against their own economic interests.
Except I think switching trades (which you would have to do since the union is all encompassing and forces membership) is a lot more onerous than emigrating, and emigration is no slice of cake.