People have the arithmetic mean in mind, but their intuition of "average" is actually the median, or even more commonly the modus.

When somebody says "Average people are X", they usually mean "Most people I've encountered are X", which is the modus. Hell, I just did that in this exact sentence!