The existence of the Electoral College, which was specifically designed to give slaveholding states an advantage when selecting the President.

The gerrymandering of electoral districts to maximize the concentration of white voting power.

To put a finer point on the electoral college, it was to appease slaveholding states so they would ratify the constitution. They wanted their enslaved to increase their voting power at the polls without actually giving Black people the right to vote. So the three fifths compromise let them count their enslaved as three fifths of a person. Now it obviously wasn't enough that a Black vote would only count as 3/5ths. The electoral college was needed so they could accomplish this while still having only white people as representatives elected by white voters.

There were other reasons for the electoral college, but appeasing the slave states in this way is one of the big reasons that was specifically written about and reasoned openly between the founding fathers at the time.

So when someone tells you, oh, the electoral college allows smaller states to have a say, think that through a bit. That old apocryphal explanation makes no sense really.