"You must test if you want to improve outcomes"

Have we not learned by now that people can be trained to score well on tests without really understanding the material they are being tested on?

The problem here is that there is no randomized sampling in these tests. Everyone is tested, and so teachers know in advance that their students will be tested and they know what those tests will cover and even how the problems will be stated. Teaching "to the test" is a practically guaranteed outcome when you start telling teachers that their salaries or careers will be linked to student performance on tests.

There is also the matter of failing to recognize that a student with expertise in math has a great advantage on tests in physics, chemistry, and to some extent even literacy tests. One cannot simply blame a science teacher whose students are doing poorly if the students' poor performance is the result of incompetent math education. If students are illiterate, they will probably do poorly on all standardized tests because they will be unable to read the instructions.

Are you saying that they weren't testing before? That the grades I received all throughout school meant nothing, because they weren't controlled by the government?

We had a measure of how kids were doing: The report card.

To me, that's what this gentleman is asking for. A return to previous teaching standards, not teaching to some standard written & defined at a government level by people who don't teach.

A few years ago This American Life did an excellent episode on what happened in Washington Irvine Elementary School when teachers were given more autonomy and then what happened when it was taken away[1].

I just listened to this episode the other night and the themes presented parallel this discussion nicely. The narrative presented suggests that one of the best things Washington Irving did was to have carefully constructed report cards that measured what specifically was being taught to students. The other thing was removing poorly devised standards.

That being said, at the end of the episode, one of the teachers interviewed, when asked about whether the temporary success of Washington Irving was due to some hard to measure effect of having good teachers at the right place at the right time had this to say:

> It can't just be magic. It can't just be fluky things landing people in the right places, and the chemistry being right. It just can't be that. What's the hope for the rest of the schools? What's the hope for Irving in the future? What's the hope for any school, anywhere?

The apparent tension between a need for measurement by standards and a need for teachers' autonomy was palpable and heartbreaking. It is well worth a listen.

As for a solution, I can't help but wonder why there is such a focus on making and enforcing standards at higher levels of administration. I imagine it gives a feeling of productivity and control. I wonder if it would be more useful if those bodies focused on studying and informing. Of course studies are harder without standards but it might be worth expending the effort, since it would give a more natural view of what is effective in promoting learning.

[1] http://www.thisamericanlife.org/radio-archives/episode/275/t...

"We had a measure of how kids were doing: The report card."

Report cards are as deeply flawed as standardized tests. If I were to tell you that I received a D in middle school algebra and an F in English, would you assume that I found algebra challenging and that I had literacy problems when I was 12? If so, you would be utterly wrong: I was beginning to understand limits and integration was I was in 7th grade, and I was reading at a "post-high-school level" according to my school.

The reason my grades were not even remotely related to my aptitude was simple: grades are not meant to reflect aptitude. Aptitude gets you a D- if you do not turn in your homework, even if you did not bother with homework because you were busy learning more advanced material. I am neither the first nor the last person to have been given grades that did not reflect my understanding or talents in any subject.

Report cards try to give a one-dimensional view of a student's academic performance, and in doing so they favor a particular kind of student. Obedient students who do exactly as they are told and nothing more, who only learn what their teachers expect them to learn, receive the best grades. Second best are those who try really hard to be compliant but who lack the talent or understanding needed to answer every question to their teachers' satisfaction. Last place is reserved for students who either do not care about being educated or those who care but fail to properly follow instructions or fit into the neat view of people that American teachers have.

Our education system is meant to train people to follow directions to the letter, and the switch from report cards to standardized tests has not changed that.

"Obedient students who do exactly as they are told and nothing more, who only learn what their teachers expect them to learn, receive the best grades."

I always completed my homework, and then went on to learning the more advanced things. It was so simple for me that in the majority of cases, completing it took me very little effort or time.

However, I'm sure most of my teachers would laugh if they heard me called obedient. I'm sure they'd have loved to have flunked me if I'd ever given them the smallest opportunity to do so.

It's quite easy to learn what you like, and have the best grades as well. The really annoying thing is running out of homework to do and then getting busted for reading a book quietly instead. (1000 page asimov novel being somewhat more obvious to spot than a page of algebra homework)

(That, and having a 105% in a class and still not being allowed to go back to sleep in your seat at 8AM. Zzzz.)

Oh, I see now. You are advocating we use psychic powers to know that a student didn't turn in his homework because he was off learning more advanced topics. You're right, we should get started on developing nation-wide psychic tests.

No, I am saying that either people need to recognize that grades are not a reliable indicator of ability, understanding, or aptitude, or else teachers need to start assigning grades that are a reliable indicator of such things. Right now, we have grades that are interpreted to mean one thing, but in reality mean something else.

Your comment starts with the assumption that having grades that are based on whether or not homework is turned in is how things should be. My point was that teachers should not be giving bad grades to students who did not do their homework, they should be giving bad grades to students who do not demonstrate an understanding of the subject.

> The report card.

Grades are assigned in three ways

1) Relative to the skill your teacher believes you have

2) Relative to the rest of the class

3) Absolutely based on what the teacher individually believes constitutes mastery of the material.

The report card is almost meaningless.

I am not sure where you went to school, but in the New York City public school system, report card grades followed a formula of roughly the following:

  65% exam scores / written essays
  25% homework
  10% classwork

A 65 was basically a D-, and anything less was an F. In other words, a student who receives perfect scores on all exams or written essays (depending on the subject) could receive a D-, as though they had only the most minimal understanding of the subject matter.

So while the report card is meaningless, it is meaningless for a different reason than you suggest. It is meaningless because it has nothing to do with aptitude or education and everything to do with obedience.

These are orthogonal issues and thus can both effect the value of a report card. Beyond that, I agree with your point.

Although you don't mention it, I feel that this post strongly implies that the teachers can't be trusted to exercise professional judgement. What's the point of even requiring teachers to meet standards if you believe that they can't properly evaluate their students?

That's orthogonal to my point. Teacher quality or competence in assessing quality, doesn't have anything to do with whether they assign grades in the ways I've enumerated. I've had absolutely amazing teachers that have graded in those three ways.

Even if they're good judges of quality, it's still a matter of how they determine your grade. Is it good quality relative to what they think your skills are? Is it good quality relative to the rest of the class? Is it good quality relative to what they believe constitutes mastery of the subject? And some teachers curve and some don't.

And I haven't said whether I think teachers should be required to meet standards. I don't even know if it's possible to create good standards.

Like most everything else, it all comes down to execution.

Typically when educators critique "testing" the critique is in two parts:

1. Why is there so much emphasis on raw factual knowledge - which is easily tested - and so little emphasis on analysis and critical thinking (points mentioned in the article).

2. The amount of testing and test preparation conducted in schools has now reached a point where it is materially effecting the amount of actual teaching that is being done.

If a school or district fails a national level test it typically kicks off a spiral of district level tests, practice tests and separate test preparation which ends up putting the kids further behind.

Any teacher worth half their salt will know how well the kids are doing.

It takes literally 30 seconds of looking at a piece of student writing, for example, to have a pretty accurate assessment of their writing ability.

The problem is that we assume we need to test students, and not teachers.

If teaching is the problem, why are we testing students?

We don't blame bad code on the users, do we?

The second problem with your statement, is the unanswered question of who is doing the testing, and who is fiddling with the knobs. Not an easy question, but one that can't be ignored.

>The problem is that we assume we need to test students, and not teachers.

>If teaching is the problem, why are we testing students?

And, err, how do you propose to test teachers?

Well, that should be the job of the principle in charge of the school. One of the big problems is that it is very difficult to fire bad teachers, even if it is obvious they are not performing well. Right now, union rules make it very difficult for a school principle to properly do their job.

This isn't 'automated' accountability. But it is a real answer, not the short-cut that many people are looking for with standardized testing.

There seems to be a common temptation to chalk student performance solely up to teachers. This despite the fact that there are plenty of things that are out of teacher's control that strongly affect how well a student performs. Here in Maryland, there have been a bunch of recent articles about an alarmingly high percentage of students coming to school without getting adequate nutrition outside of school hours. This has a huge effect on concentration acquisition of new material. In general, socioeconomic and cultural differences between communities have very strong effects on schools; teaching in a school district with highly involved parents gives a huge boost to test scores vs. districts with largely uninvolved parents. Same for rich vs. poor districts, rural vs. suburban districts, etc.

Also, what makes you think school administrators have some answers that teachers don't when it comes to how to do the job, or have any idea, really, how to fairly and correctly evaluate teacher performance. Similarly, why do the same people who routinely call for teachers to be fired for poor performing classrooms never seem to want to hold administrators and school boards to account for their poor performance? Why do state and local governments who cut education funding year after year get a pass?

I think it's a comforting world view to think that if we could just bust the teacher's unions and start firing people, we'd get real results. But it ignores the fact that the system is broken in may more ways—and in many worse ways—than just the teachers.

Are you assuming I'm chalking student performance solely up to teachers?

Are you saying that a teacher's supervisor (the principle) shouldn't be in charge of evaluating their performance?

Are you assuming that I think administrators and school boards shouldn't be held accountable as well?

Are you assuming that I'm saying firing people will fix the system?

If we can't overcome a problem as simple as firing bad teachers, how can we address so many of the other issues that face the school system? Does the fact that other problems exist mean we should ignore this problem?

Well, in your parent comment you say "The problem is that we assume we need to test students, and not teachers." So, that does seem to be what you were saying.

Of course a teacher's supervisor should be in charge of evaluating them. But they must be competent to do so and they should do so through evaluation of individual lesson plans and classroom observation, not the performance of the teacher's students on a standardized test. That's not a straw man by the way, there are a lot of people in this debate who think that's an acceptable metric for making decisions on compensation and hiring and firing [1]. Whether you are one of them, I have no idea. But the argument you make above is one I have encountered many times, and typically its adherents are quite myopic about the culpability of everyone else in the system.

Of course there should be a way to evaluate teachers and hold them accountable. And there should be a way fire them if it's called for. But it has to be equitable. And it shouldn't hold teachers hostage to factors they don't control. But even granting you that, I haven't seen any evidence that this is going to lead to much an improvement in the educational system. There are much bigger problems here than "bad teachers".

[1] http://www.epi.org/publication/bp278/

If you randomly assign students to teachers (which is already more or less the case) and each teacher has a large enough sample size of students (which is already more or less the case), aggregating the performance of all of a teacher's students will say more about the teacher than about these confounding variables, since all the other factors would be shared between all teachers at the school.

Of course there are other problems to be solved. But the fact that there are no effective controls whatsoever on teacher performance, and that teaching is not a highly selective profession, is still a problem.

I address this further down thread, but in a nutshell: there are effective, fair ways of evaluating teacher performance [1]. And I'm all for them. Standardized test performance is not on of those ways. Furthermore, in a lot of the schemes currently being put into place, teachers are not evaluated by test scores at the school level, but at the district or state level. Teacher evaluation is necessary, but it's not easy. Trying to shortcut things by asking standardized test score to do more than they are capable of (which is not much) is exactly the kind of thing that drives good people out of teaching.

[1] See: http://www.epi.org/publication/bp278/ and http://www.washingtonpost.com/blogs/answer-sheet/post/meet-a...

That sounds fine for intra-school comparisons, but as far as I know, the teacher comparisons are inter-school, which is a whole lot less random in student assignments.

Ok, that makes more sense, but isn't how you phrased it initially. (No one would call that "testing teachers").

You say test but I think you mean measure. You can't improve unless you have a way to track improvement.

There are many ways to measure though and tests are just one of them. They also happen to be the easiest and most understood, which is why they get used in schools and by use officials to track and rate how well a school is doing.

I agree we need to test but I think the problem is what we do with the test scores. It should be a natural goal for a school to raise it's test scores as time goes on. Learning the correct way to do things and improve. The main issue I see is how funding is provided to schools based on the scores. It makes everyone focus solely on the scores because each administer is trying to get money to run the school which is based on how they test. Provides the wrong kind of message. We should be passively looking at these scores and sending additional support to schools that are low not basing how much money the school gets based on them.

"It should be a natural goal for a school to raise it's test scores as time goes on."

With many of these tests, especially good schools are hitting the ceiling of how well it's possible to do. If you're scoring a 99% every year, how do you improve?

The problem isn't testing, its tying funding to the result of these tests. It is this fact that has caused serious turmoil in this country's educational system. It must be stopped.