This is discussed extensively, midway through the article, in approximately four paragraphs:
Students were better than trained adult observers at evaluating teachers. This wasn’t because they were smarter but because they had months to form an opinion, as opposed to 30 minutes. And there were dozens of them, as opposed to a single principal. Even if one kid had a grudge against a teacher or just blew off the survey, his response alone couldn’t sway the average.
“There are some students, knuckleheads who will just mess the survey up and not take it seriously,” Ferguson says, “but they are very rare.” Students who don’t read the questions might give the same response to every item. But when Ferguson recently examined 199,000 surveys, he found that less than one-half of 1 percent of students did so in the first 10 questions. Kids, he believes, find the questions interesting, so they tend to pay attention. And the “right” answer is not always apparent, so even kids who want to skew the results would not necessarily know how to do it.
Even young children can evaluate their teachers with relative accuracy, to Kane’s surprise. In fact, the only thing that the researchers found to better predict a teacher’s test-score gains was … past test-score gains. But in addition to being loathed by teachers, those data are fickle. A teacher could be ranked as highly effective one year according to students’ test gains and as ineffective the next, partly because of changes in class makeup that have little to do with her own performance—say, getting assigned the school’s two biggest hooligans or meanest mean girls.
Student surveys, on the other hand, are far less volatile. Kids’ answers for a given teacher remained similar, Ferguson found, from class to class and from fall to spring. And more important, the questions led to revelations that test scores did not: Above and beyond academic skills, what was it really like to spend a year in this classroom? Did you work harder in this classroom than you did anywhere else? The answers to these questions matter to a student for years to come, long after she forgets the quadratic equation.
Fortunately the authors weren't so careless — that concern is obviously key to the repeated cautions about assigning too much weight to such tests. I'd like to believe anyone setting policy would be similarly slow to jump to conclusions but this is a field notoriously prone to chasing fads and quick fixes…
For example, in one programming summer camp I led, I've had kids come to me at the end of the 10 days and tell me that I was their favorite educator and a great director because I was tough but fair and expected them to learn and progress, while some other educator was lame because even though he was super nice and laid back, they didn't learn anything with him.
Now of course, during the camp itself, I've overheard more than one kid telling another that I was super mean and terrible. But in the end, most of them were vocal about how they appreciated me. Kids in general tend to be very aware of and appreciate fairness, consistency, and holding them up to standards and pushing them to get better.
Cases like that are not the exception, but sadly not the majority either (the specific example I cited was with French "gifted" kids between 14-18 years old). So it would really depend.
As a recent high schooler, I've seen kids who absolutely hated their teachers because they gave them bad grades because they challenge their students. However, this specific teacher was probably the most influential for those who appreciated him.
I'm just afraid that the majority will figure out the system and use it as a weapon, as was mentioned.
Only if it's used as part of a (simplistic) algorithm.
And besides, why would students want to fire the good teachers? That doesn't really make sense. (As someone who was sent to the principal's office on a semi-regular basis.)
Later, whenever I met a new teacher (at university, in the workplace) lessons always seemed easy to me. I had to work hard for a good grade at high school, and her tests demanded precision and attention to detail. I started to appreciate her more.
Students are likely to prefer teachers with a pleasant attitude and lower difficulty level. A teacher easy to distract with an off-topic conversation. But at the end of the day, a teacher's job is to teach. Sure, a great teacher would combine that with charisma, ability to inspire and natural charm, but nobody's perfect.
Win!
Part of the reason you see so much backlash from teacher's unions against these initiatives (despite the fact that virtually all teachers acknowledge that some are better than others, and have a pretty fair idea who the worst ones are) is that they don't see this as a way to fairly evaluate teachers and improve the quality of education. Rather, they see it as a weapon to be wielded by school districts, school administrators, and angry parents ("Mrs. X told little Johnny (age: 17) where babies come from!" "Mr. Y is a member of $MARGINALIZED_SUBGROUP!" "Ms. Z believes something that is the current scientific consensus" "Mr. Q gave my son an A-") against unpopular teachers. It is an unfortunate fact of life, and even more unfortunate that so few are willing to admit it (especially here on HN, where the quality of discussion is usually above average).
We would probably have much greater buy-in from teaching unions if these evaluations were perceived as something other than politics as usual. Unfortunately, the proposals all seem to keep a some level of human involvement in the firing process (so only the very most outstanding teachers are immune to the usual political pressures). It's useful at this point to recall a bit of history: Teachers today are so hard to fire because they have tenure. Teaching unions successfully forced tenure the tenure system on school districts because school administrators of the past abused their authority to fire good teachers who happened to be politically unpopular.
I'm also not so sure that everyone would be so willing to honestly critique their teachers at the end of the year, either. A lot of kids come to like their "bad teachers who lead fun classes" almost as much or more than the teachers who do their jobs well. Kind of like how many kids are unwilling to "snitch" on kids who have wronged them, even when guaranteed anonymity, I think a lot of them wouldn't want to get their teachers in trouble, even if they weren't doing their job properly. In my experience with such teachers, even poorly behaved students would act extremely respectfully when the principal came around to observe. Then again, maybe they were just afraid of getting on the teacher's bad side.
Even worse are instances where the administration knows fully well that a teacher leads a "joke class" and keeps them around to funnel the "bad kids" into. I grew up in a school system that straddled the border between a middle class college town and Section 8 housing packed with families ruined by the collapse of unskilled industrial labor in the area. I have no doubt that a large number of the elective courses existed solely to fill credits for disruptive students and keep them from causing problems in the "real classes" that the middle class and brighter/better-adjusted lower class students took. It didn't matter if the class actually learned anything, the school was just desperate to raise their graduation rates.
I do think these evaluations can help good teachers improve and possibly help to identify people who don't deserve to work as teachers, but they can't solve fundamental problems like "the early education and home environment of these students has failed them, and now the administration of their high school is ready to roll over to keep them from dropping out."
I was a dick back in high school. The hacker I was back then would have figured out exactly how the testing and metrics were set up (public information) and organized a union of students to manipulate it. I can't do much with standarized test scores, they reflect on me. But a teacher quality survey? That's just a weapon.
Things like this make me wish that we had some kind of Hacker in Chief, to figure out how to circumvent new systems before they get implemented.