Absolutely true, but I'm not sure that's really a point of confusion. Almost everyone would agree that smart people aren't automatically good at athletics, or poetry, or woodworking. If you've seen people say that kind of thing I agree they're being silly.