With freedom of association, a union and employer could reach a contractual agreement to require that all employees who enter into an association with the employer must pay into the union, regardless of membership status. Freedom of association allows an employer to reject those terms, of course, but if they do enter into that agreement then that's the terms of engagement that must be adhered to.
The government lording over saying "You can't do that", even when both parties are agreeable to the terms, removes necessary freedom. If the employer wants to associate with you without you needing to pay into a union, they can reject those terms on their own accord. If they do enter into that agreement, they have the freedom to not associate with you if you don't like the terms (as you the freedom to not associate with them).
I, an individual, could equally say "I hate that sokoloff guy and I'll work with you only if you agree to never hire him." and at that point they have to decide if I'm worth hiring under those terms or not. If they want to keep you open as an option then they can say "Nope. Sorry. Best of luck in your future endeavours."
That combination seems to increase (or at least leave unchanged) my freedom of association. (Others who do wish to associate with that union are still free to do so, of course.)