I'd go with subjective judgment by human observers, which is fallible but I think reasonable enough, and harder to game, when you're looking for large differences. We're not really talking about distinguishing 50th-percentile from 60th-percentile, but trying to pick out "absolutely horrible at teaching" (get rid of them) from "fine at teaching" (keep them) and "amazingly great teacher" (give them raises/bonuses). I think principals might actually have a good estimate of that even today: the worst teachers, at least, are usually something of an open secret. If principals ideally stuck around longer (instead of the current musical-chairs that seems to go on), they'd have even better estimates.

Sort of the same way that companies do it, really. Engineering firms, or good ones anyway, rely more on subjective judgment of management ("so-and-so is a [terrible|great] engineer") than they do on code metrics. I think they'd have some of the same morale problems if they were heavily metrics-based as well: imagine if every year your company ran a script over their VCS+bug-tracker with whatever metrics were currently in place, and fired everyone who scored in the bottom 5%.

Lines of code produced is a terrible metric of software progress and especially software quality. By comparison, a well constructed test is an excellent metric of learning.

Also, have you ever written software for the government? I think if you had, you'd yearn for an objective evaluation procedure as accurate as standardized tests. For your proposal to work, you'd have to carefully set up incentives for the judges and try to figure out how to keep the politics to a minimum.

If "teaching to the test" is a bad thing, that just means you have a bad test. With varied enough questions, the easy way to prepare for a test is to understand the material. I agree that teachers shouldn't be judged directly by the scores of their students, because not all students have equal background and ability. But any time you have an opportunity to measure something objectively, jump at the chance. That goes for software, too.

Well, "well constructed" is the tricky part. It's not even clear to me what one would look like. K-12 schooling has a lot of goals, most of which don't involve literal transmission of information, but boil down to something like, "we want as many kids as possible to be normal members of society, with decent jobs, some sort of civic engagement, not engaging in criminal or shady behavior, and able to handle their affairs". Ideally we'd also like some subset to be brilliant, although it's not clear how much K-12 education influences that.

You can judge most of the goals in retrospect, say 20 years out: what proportion of kids ended up in jail, what proportion ended up with good jobs, what proportion are informed citizens capable of reading the news and participating intelligently in the political process, what proportion are able to handle their personal finances, nutrition, health-care, etc. successfully, what proportion won prestigious awards or started successful businesses, and so on. But writing a test for 18-year-olds to take that predicts all that accurately is pretty hard, and as far as I know, no tests have shown even plausible evidence of their predictive power in that respect, especially when you control for demographics.

Again, I'm not saying that 100% of teacher evaluation should be based on tests. If you want to evaluate teachers based on the moral character they are building in students, fine - that's not what I'm advocating standardized tests for. They're for measuring "book learning" and they're pretty good at it.

>If "teaching to the test" is a bad thing, that just means you have a bad test.

Look at it this way: We do not want students to all know the same things. We want to increase the amount of learning, but ideally, only 30-50% of the knowledge kids learn should be held in common. In a perfect world, the remaining 70% would be different from classroom to classroom. Varied experience is crucial to making our society function better.

So if we take varied experience as a value we want to encourage in our classrooms, you cannot have a good standard test, because the standard is more amount and quality of knowledge than a predefined notion of what knowledge is necessary to get by in the world.

And likewise, because this is how I value it, I would prefer that standardized testing be around one-third of the evaluation score, if not 20%. I want teachers and school administrations to be free to take big risks like teaching everyone computer science instead of math - because teaching a well-thought-out course of study is not a risk. Classrooms should definitely do instruction in ideas of the past (history is always vital), but more than anything they should be incubators for new ideas.

Well, by your own admission, 30-50% of knowledge should be held in common. In early grades, the number is probably more like 90%. By high school, sure, there's plenty of room for diversified knowledge of literature and history, but there's also a core of math, logic, and reading that everyone should know. Why not test this stuff? Standardized tests are objective.

Now, the current system focuses on meeting minimums, and most standardized tests quickly hit a ceiling. I'd much prefer to see tests that attempt to gage the entire breadth and depth of student learning. Practically, that's going to require different questions asked of different students, or you'll end up with a 10,000 question exam. But I still consider it standardized testing. The SAT and AP exams are great - we should do more of that earlier.

>Why not test this stuff? Standardized tests are objective.

Sure they are, but to what degree do they really reflect on the teacher? A teacher can be doing a fantastic, patient job with a student, and make zero progress all year with a student. That work is meaningful, but its effect could take years to show up. Can we measure this? Sure. But it will be very error-prone, and ultimately less effective than just asking the other people in the building who is worthless (everyone knows except the worthless themselves.)

We're not looking to find the best and brightest - we as a society have made the decision that we're not going to pay that much. We just want to get rid of the unequivocally awful.

Simple solution: create a set of standardized tests, on Math, History, etc, and require students only to pass a subset of them.