I might be able to help by pointing out that a technology that depends on random sampling of small components of "intelligence" or whatever is going to be totally screwed up if the sampling is no longer random and now the outcome of the test depends mostly on the amount of money spent on prep classes.

From manufacturing I can predict a lot about the quality of something in general off just one graph of the distribution of length of one part compared to other manufacturers graphs of the same single measurement on the same part, but if the mfgr "cooks the books" by optimizing one measurment of that part to perfection while taking away labor from other areas of part quality, then my comparisons will be useless, especially when comparing optimizers vs non-optimizers.

This is quite likely the case with school children, but often not the case with, say, industrial processes. If, however, you make someone's job dependent on meeting some particular target for that process, then a feedback loop is created that can lead to the kinds of problems being discussed.

If the feedback loop isn't there, then the metric will remain as useful as it ever was. So the point is, be wary of unintentional feedbacks in your system, not some grandiose rejection of the possibility of optimisation.