Everyone is hearing the same sound, and classifying it as a different word. Just as everyone is seeing the same color, and classifying it differently.

I'm not sure everyone is hearing the same sound. Some people hear higher/lower frequencies better than the others. The signal that the brain receives is completely different from person to person. The sense of color is probably more consistent in that respect, although some people see less colors (colorblindness), and some see more (tetrachromacy).

What would make you say people don't perceive higher/lower frequencies of light better than the others, the way you claim about sound frequencies?

Yes, if you simplify the discussion by removing important differences - which is probably not the best idea when talking about classification.

There is a significant difference between people classifying things differently when they exist right on the boundary between two classifications (ie two implementations of the same classification method will frequently disagree), and classifying things differently when the classifications are nowhere near each other (ie you’d hope an objective method has a significantly lower error rate).

The former is about the accuracy and precision of your measurements. The latter is about how you interpret them. Both are important, but are also different subjects.