Yoga only existed as a religious practice until the 80's when the physical part was separated. It's not a surprise that people who grew up before the 80's don't automatically think of the non-religious version. It's hardly "ancient".