What you're missing from this observation is that exercise doesn't extend one's lifetime that much, but just improves the general quality of life, meaning that 80-year olds that are/were active in moderation now enjoy a bit better QOL [1].

[1] https://academic.oup.com/biomedgerontology/article/56/suppl_...