Yes, any ranking on subjective criteria is nebulous. But it's definitely not uncommon to rank school performance. State testing is the easiest data to use because it's typically public information.

At the time I went to the school, there was a national ranking that used a variety of factors, including number of IB/AB courses per student, (P)SAT/ACT scores, and GPA.

I absolutely don't think the ranking was meaningful, but it at least says something about how many of my classmates had affluent parents who put a lot of pressure on them to succeed.

Now what's the matter buddy, ain't you heard of my school? It's #1 in the state!

People have been true to their schools since at least 1963.