> There are a lot of great doctors who didn't decide to pursue medicine until college, or maybe even later, and as a whole the field seems fine with that.

Med schools are increasingly selecting for that, in fact. Second careerists tend to do better (because they're adults), and have less burnout (because they're more likely to have made a knowledgeable adult decision about what they're getting into, rather than being disappointed reality didn't live up to fantasy.)