Regardless of the semantics of elite and what she thought it meant, she knew that you went to a school associated with kids of politicians and rich people.

That’s consistent with the general definition of elite. Maybe what she and you were thinking of was “smart.”