I think you run into the barrier much faster in something like sports because the gap is so easily measurable. E.g. it just takes one 100m sprint to see how slow you are.

Whereas with intellectual work it's often hard to assess the gap, not least because the further you are from closing it, the less understanding you'll tend to have of how hard the remaining parts are.