That having been said, for this particular example (life expectancy) it's typically a bit better to look at life expectancy at age x, often 5, 10 or 15 are chosen.
It's because infant mortality used to be sky-high, as 25% (one in four) in 1900 and 16% (one in 6) in 1950 worldwide, to < 2.9% today, with the best countries now around 0.2%.
If say one in four humans becomes virtually zero years old, that skews the average down a lot, e.g. from 65 to 40. But it doesn't mean that the average person who makes it through the first few years would've died at age 40.
If we look at improvements of life expectancy for those who already raeached age 10, it looks like this and is a pretty straight line with fewer indications of slowing down. https://ourworldindata.org/grapher/life-expectancy-at-age-10...
Why did we make more progress before and less later? We have better tools, computers, and software. We have more people, both in the US and coming in from worldwide along with a greater portion of the population being college educated and free to work on scientific and engineering problems. We have better developed theories of biological, chemical, and physical systems. And yet, when we knew very little, had little resources, and few people working on the problem - we made huge strides. Now, not so much.
Maybe this issue is that we've already plucked the low hanging fruit. That doesn't seem quite right to me though. Even if we have, we should be equipped, via the advantages I referenced earlier, to pick higher fruits.