I think the issue with meritocracy is when people assume meritocracy exists and then operate completely ignoring that there is no meritocracy. See- the original use was mocking the idea that you can assume you can judge someone's apparent merits in a vacuum.