"Terrible at geography" is not binary state; it may well be true that Americans (on average) are worse than most others, while having absolutely brilliant scholars.

High standard deviation.

Plus I believe the real stereotype of Americans is being ignorant of geopolitical issues, dumb bigoted nationalists. Nobody really cares whether you draw some obscure peninsula correctly or not.