Is it the same test (same questions) or a different test? I would guess same type of questions.
Also what if the students are already top performing and don't improve but actually get a slightly worse. (As in they all get 95% and then, well a few get sick and skip a few months from school and now the class gets 94%).
I guess I am still confused at how average expected improvement is supposed to work reliably across all those population models.
Do we assume the school-wide population of students is uniform enough to calculate a meanigful average improvment from it which can be applied very classroom individually.
There are a lot of school districts that have very mixed populations (economic, ethnic, cultural) backgrounds.
Same about the state. Some states districts that maybe very rural and undeveloped mixed in with a metropolitan area some place across the state.
What it comes down to is, what's our other option? Does it even matter when it's nearly impossible to fire a bad teacher?
Here's a simple way to compare someone to the average teacher.
Two tests. One at the beginning of the year, one at the end. Take the average improvement across the school, across the district, across the state. Now apply that average school-wide improvement to a classroom's original scores in the first test to see how those students should have performed on the second test. Compare that number to how they actually performed.
I came up with this in about 10 seconds, there's obviously a better way to do things, but comparing someone to the average teacher really isn't that hard to do.
The most important part is coming up with a test that actually measures understanding. Open ended questions are usually a good place to start. Many tests I have taken really require you to understand the material.