Within a given college, it is likely to be realistic. Take U-Mich, for example (since they are the only school to reveal their point system) - being a student athlete is worth +1.0 on GPA. I.e., a student athlete with a 2.5 is on equal footing with a non-athlete having a 3.5. This means that the intellectual average of student athletes is highly likely to be lower than the average of non-athletes.
This applies to Michigan specifically. They are willing to make academic sacrifices for the sake of their athletic programs and (maybe) a well-rounded student body. Not all schools operate this way.
For example, suppose the academic ability of jocks is normally distributed about 105 with a std dev of 15, whereas non-jocks are normally distributed about 100 with the same std dev. If the cutoff for admission is 110, but athletic talent gives you a 20 point bonus, then you're your comparing mean aptitude in a sample of non-jocks, given that every observation is >= 110, to the mean aptitude in a sample of jocks, given that every observation is >= 90. Under those conditions, you'll find a 10 point difference in favor of your sample of non-jocks, even though the population mean for jocks is higher.
I think you alluded to this point in an earlier reply, but it's worth spelling out.
Either way, it is worth pointing out (perhaps to the choir) that the stereotypes of dumb jocks and fat nerds are not realistic, if they ever were. Strong minds correlate with strong bodies.
Which reminds me, I need to get back in my routine, and go out there and run today after work. :)