And then you watch student performance get gamed by the teachers. There are all kinds of ways: turn a blind eye to cheating, move poor performers to others schools (including by getting them to graduate when they shouldn't), alter their tests (this is already being done under NCLB according to several news reports), ...

You're better off by having skilled teachers rate the other teachers. People aren't as easily gamed as systems of rules are.

Tests are given by independent proctors and designed by an independent agency. In much the same way, the DOD rates Halliburton's performance - it's not great, but it's better than Halliburton doing their own performance ratings.

move poor performers to others schools (including by getting them to graduate when they shouldn't)

Won't help with VAM. If you get rid of the poor performers, the predicted value of the people remaining will increase. Thus, you just made your job higher - the bar has been raised.

People aren't as easily gamed as systems of rules are.

I don't even know how to respond to this.

Yes, independent ratings can help, but at some point, if you can't trust anyone inside the system, you're screwed. And your results are exactly as good (or bad) as your ability to do ratings. Fancy math won't fixed a flawed premise, for example.

We've been going for complex systems and simple people. Having good people and relatively simple systems, however, is the optimum from what I can see.

I'm not opposed to the idea that accountability is needed, but the idea that you can go up to a troubled institution, make everyone's job harder, and then expect improvement is unrealistic in any industry. I'm not a teacher, but I've seen the effects of management like that first hand.

Always taking the easy solution when managing something is not effective.

>> People aren't as easily gamed as systems of rules are. > I don't even know how to respond to this.

Obviously, I thought it implicit that I was talking about competent and experienced folks. Yes, if you get people who know nothing and put them in charge, you will have terrible results. They won't improve even if you give them a long list of rules to follow, which is how bureaucracy usually ends up.

Would you prefer to do things the way the educational system does? Throw money at Halliburton and hope for the best? No outside auditors or performance measurements?

Those who are competent ought to be able to recognize each other.

But if you had a kid, and you had two teachers, both of whom regularly got classes in the 50th percentile, and one teacher regularly had kids leave in the 40th percentile, the other in the 60th, which would you want? In fact, L.A. is already now in effect conducting the experiment: http://www.latimes.com/news/local/la-me-teachers-value-20100... and parents are responding accordingly -- for good reason.

Your concerns are valid, but they are mostly misguided if the alternative to them is "doing what we're doing now." See here: http://www.marginalrevolution.com/marginalrevolution/2010/08... for more; "We cannot simultaneously claim, however, that teachers are vitally important for the future of our children and also that their effectiveness should not be measured." Also see the list of education-related articles I compiled here: http://jseliger.com/2009/11/12/susan-engel-doesnt-get

If your argument is "nothing could be worse than the way it is today", I strongly disagree. More than 70% of Americans are satisfied (or better) with their own local school. High-stakes testing can easily make schools worse by disrupting curricula and damaging incentives for teachers. If we're going to focus our efforts on the school systems that are in crisis, there are probably better interventions.

Measuring teacher effectiveness is a real problem that needs real work, but in the interim, primum non nocere.

Also, the process MAY or MAY not have harmed other students -- there is no reason to believe it would, that I can tell -- again, please respond if there is.

How could that ever go wrong :)

EDIT: More seriously, if it's already impossible to measure, I fail to see how more complicated metrics would do anything but do an even worse job of measuring. There might be exceptions to this rule in a few cases but it's generally the way things work.

If you have low quality students, who are only expected to score at the 25% level, and you boost them up to 35%, you win. If you have high quality students expected to reach 75%, but the only achieve 65%, you lose. Do you believe it's really that hard to predict that Asian kids in 2 parent $250k/year+ households will score higher than black kids in 1 grandparent $10-20k/year households?

If you truly believe the effects of education are immeasurable, perhaps we should stop wasting money on it. We don't spend $859 billion on homeopathy for much the same reason.

Developer productivity is really hard to measure with universal statistics -- every attempt to do so has failed miserably and usually made things worse. Does that mean that nobody should spend money on development?

Incidentally, measuring developers is done routinely. Every bank (except maybe Citi) has a set of procedures and guidelines for outsourced development. These procedures make the process fairly predictable and they reduce costs/dependencies.

Much like outsourced IT, teaching is repetitive grunt work. It can and should be defined and measured.