Used to be really into language learning, this topic comes up a lot there. I'm of the opinion that while the intrinsic ability to learn does drop in some ways [0], a larger part of the difference is simply the fact that we don't put in the same effort. In first language acquisition this is pretty obvious. The incentives are all there. And I'd argue, in a much subtler way, similar mechanics persist in many fields of learning.

Diminishing returns, lower tolerance for messing up, a lot of other going concerns, I could go on and on. The point is you're probably not putting in the same effort.

[0] say as we age our hearing ability predictably diminishes, accounting for some of the difficulty in learning accents etc. But the majority opinion, I believe, over-attributes the difference to reduced plasticity and the likes.