Actually caste isn't a binary classification. And reservations don't apply only to one caste. Many castes get reservations.

The remainder bulk of the people who don't get reservations fall into this category called 'General Merit'.

Now the system fills up the reservation quotas first, leaving leftovers to general merit. General merit has way more candidates and a larger section of people who score above reservation people. The net result is you end up denying top performers seats, so that you can fill reservation quotas.