Not at all. Free speech is not binary.
More is better, & the cultures that have consistently had more speech tolerance than elsewhere - like the US, in its formal documents & actual practices – are showing that free speech is a more central value to them.
If you had a country where 95% of individuals had absolute freedom of speech but 5% did not, is that more or less free than a country where 100% of people had near-absolute freedom of speech?
What about when those percentages collide? When the free speech of the haves results in the suppression of the have nots? And historically the freedom in US has been in theory and not in practice what with how it's been used to hunt down perceived communists, minorities and dissidents.
The best possible evidence against the idea that more == better is played out in the various sites that try to have absolute freedom of speech. The end result is that you get a large amount of people that others either do not want to associate with or get chased out by the rabid angry mass. Websites and forums form a sort of microculture that prove you need some guardrails to prevent the bus from going off the edge.
But yes, more free speech – in terms of being able to honestly express your opinions, without the threat of punishment by the state – is better. From any particular state of "absolute" or "near-absolute" free speech, giving some people extra freedom to speak without state punishment is an improvement.
We were talking about the US, as a polity/legal-system. Private forums can and will set their own standards. But the US public-sphere, and what the government can get away with punishing, has for centuries had world-leading free-speech guarantees.
Race is a far less salient variable affecting speech, in the US, than income or education.
Many of the US's most fearless speakers, often relying on our legal protections or cultural appreciation, especially with respect to criticizing the powerful, have been from categories of people that are otherwise economically disadvantaged.
And, their practice of free speech, aided by the underlying high value from the culture & legal system, has helped to correct those other inequities.
The last 2, gets a bit self-congratulatory. Jim crow laws (until 1965). Emmet Till didn't get much rights to free wolf-whistling https://en.wikipedia.org/wiki/Emmett_Till And plenty more.
"If Americans have not consistently enjoyed equal free speech [...] then that's more a knock on the US' value of equality"
If you don't have 'equal free speech' then you don't have 'free speech', full stop.