I DO get annoyed when I see those scores, but in a lot of cases even having a leaderboard is just something that was introduced in the game just to be more "social" and less because it makes sense in that specific game.

And yes, it's not ridiculous, on the contrary, it's perfectly understandable.

Of course, these kinds of questions depend a lot on the game in question, and I think they don't have a definitive answer :)