You're right. It does seem ageist, when seen in isolation from the outside. It seems to suggest that the field of mathematics only and solely cares about the work of young mathematicians.

Yet, is it perhaps possible that this position may signify a person possessed of a wonderful opportunity to become more educated? The Fields Medal is not intended to be the capstone recognition of a mathematcian's career any more than the Clark Medal is in economics. It's intended to award outstanding achievement for a younger contributor in a field that infamously tends to worship the more senior. The IMU awards other medals as well, including one for lifetime achievement.

In short, you're completely right to be bummed out! But perhaps you could take this glorious opportunity to learn more.