It is good sense to appoint individual people to jobs on their merit. It is the opposite when those who are judged to have merit of a particular kind harden into a new social class without room in it for others. - the man who coined the term.

Basically meritocracy can turn into classism, and due to accumulated advantage it can entrench. Everyone has a chance in theory, but in practice advantage compounds.

There's also the simple human welfare problem that vicious never-ending competition every minute of your life is exhausting & demoralizing.