Yes, anyone neutral or right of center knows all about that unreceptive ear.
Without knowing that, broadly speaking: freedom of speech has never implied freedom to have people agree with you. And freedom of the press implies a lack of obligation to publicize information the owner of the press doesn't want to promulgate. It also implies an editorial right to modify published information, ideally with explanation but that explanation is not required (the back-stop on that right being if the public thinks the press's output is trash, they don't have to care what it has to say).
Some positions, you can argue until you're out of breath. If the position is bad, the argument won't win support no matter how much breath is put into it.
The argument is simple: people should be free to include or exclude others in their private business/university/home/club as they see fit, and suffer free market consequences for doing so. This freedom was taken away by the civil rights act and the supreme court.
History suggests the market can't solve everything. History also suggests the majority is not always right, which is why the US government is structured as a system of checks and balances and not a simpler-to-implement mobocracy.
But (like any business owner) if a minority business owner wants to incorporate and exchange money with the public for goods and/or services, their rights are curtailed like everybody else's are regarding who they may not refuse service to. For example, a black business owner isn't allowed to kick white people out. They are allowed to kick Klansmen out (as per federal law; states may place additional restrictions).
It is the nature of societies that we give up some rights to protect others. The law is one long, ongoing conversation on what that trade-off looks like. And in the specific case, I'm of the opinion that we tried it the other way (refraining from curtailing the right for a business to refuse service universally) for at least a hundred years and found that it didn't make a good society.
(IANAL disclaimer) The freedom of association has been implied by analogy to be protected by the explicitly-enumerated freedom to peaceably assemble in the First Amendment, in some specific cases. But it's a loose protection by analogy; the letter of the First Amendment doesn't even mention a freedom of association. The SCOTUS has not ruled that the freedom of association extends to a freedom to exclude in public accommodation; in layman's terms, the reasoning is "You have the freedom to serve as many people as you want, but that does not imply a freedom to not serve other people if you want to operate under the corporate charter that the government grants."
This overview describes things much better than I can. https://constitution.findlaw.com/amendment1/first-amendment-...
> The court was wrong and will be wrong again.
No doubt, but I don't think you'll find a receptive ear (in the public or in legal circles) to the notion that the Court has been wrong regarding upholding the Civil Rights Act.