I think the argument is that it's far easier for wealthy families to game metrics that aren't being objectively measured. E.g. they can pay for all sorts of extracurriculars. They know to play up any hint of multiculturalism / diversity. Etc.

Outside of cheating, there's truly a limit as to how much gaming can go into standardized testing.