The Supreme Court also held that "Separate but Equal" was perfectly fine. They don't get things right all the time.
And the argument is that money isn't speech.
The Supreme Court's job is to decide what laws are legal, not what they think the law ought to be. Deciding what the law ought to be is Congress's job.
Do you suppose that nine Supreme Court justices might have considered and already dismissed your three-word argument in their lengthy decision on the subject?
How many of the folks who argue against the Citizens United ruling have actually read it and understand the legal basis for it?