Except that they're lying about it by classifying them as a "hate group" over ideological differences.
There is a major category difference between the Klan, out there burning crosses and lynching people, and some nonviolent conservatives out there saying nonviolent conservative stuff.
Pointing out the difference isn't violating freedom of association, it's just pointing out the difference.
I agree can completely freedom of speech means you can accuse the SPLC of lying. But it also means the SPLC can call whatever they want a hate group, and the solution to that is also more speech. So far in our little marketplace of ideas you've failed to get me to buy what you're saying.
As to the argument that it doesn't confuse anybody, it sure seemed to confuse that guy who went and shot up the Family Research Council because they were on the list.
And the answer to that is more speech. This is the more speech. Nobody is saying that we should shut down the SPLC because they publish a misleading list, only that everybody should be aware of how they're misleading people.
Or to voluntarily choose not to associate with them, which is all the SPLC is advocating people do.
Or are you for freedom of speech but against freedom of association?
In general, your mistake is confusing "constitutionally can't be excessively limited by the US government" and "a good idea".