Yes, this ^^
There is even a mathematical case to be made that there are always "more relationships out there to discover." One of the more plausible, sober interpretations of the various limitative phenomena in the foundations of mathematics discovered throughout the 20th century is that mathematics is not reducible to fintary symbol manipulation. Perhaps this is a bit loose and controversial, but the point is just that we don't necessarily need to solely lean on the observed fact that there have always been "more relationships out there to discover."
I think that a lot of the inherent beauty of mathematics (there is also the beauty of the climb, as you very nicely described it) comes precisely from the realisation that, sensible as this claim is, it doesn't seem to be true: there seem always to be more relationships out there to discover, and their hidden and surprising natures can make them all the more beautiful once uncovered.