It’s like how Oxford cloth button down shirts or cotton knit polos are sportswear, or a sport coat (it’s in the name!) or how jodhpur boots are “equestrian”, et c.

They started that way, but not so much now. Still, that’s the lineage they fall under in certain contexts.