A factoid about brains: we can attach a given type of input (say, visual input) to any one of a variety of brain regions X Y Z, and each such region is capable of learning that type of input. But these regions are unlikely to all be similar.

So even though, for example, we all seem to understand English, it's quite likely that different peoples' understanding of the various aspects of the language can arise through architecturally distinct brain regions. As you say, there is a veneer of similarity that hides how differently different people actually experience the world.