You're not wrong. Service academy seats and ROTC scholarships are competitive to get (like getting into any top-tier college), and then there's a decent amount of attrition along the path to commissioning, followed by more rounds of competitive selection and attrition for certain military communities (aviation, special operations, etc.), followed by the brutal up-or-out promotion and assignment system. So the people who are left standing are often fairly impressive.