I've heard the same thing from doctors; falling asleep in less than 10 minutes is generally considered indicative of sleep issues of one sort or another.

OTOH, ~90% of the population uses caffeine regularly, and given that it takes ~48 hours to completely purge your body of caffeine, I think that saying is only relevant to caffeine drinkers.