So let's talk about the software analogy. In software, the bar for formal acceptance usually really low. Do test cases pass? Maybe there's an auto-integration test that also must pass, and some requirements checkout QA has to do. You can write terrible code and have it pass the formal process, which is why you see a lot of terrible programmers coasting along at some of the worse companies. Some teams might have more formal requirements but that goes team by team. If you're working anywhere worth working, there'll be a much higher non-formal bar set by your manager, your team, your customers if you're in contact with them. So the incentives are there but they're personal and informal. The 'waterfall' pattern is one example of a misguided attempt to formalize the software dev process and align it with goals.

How would software devs feel if some perverse, untouchable bureaucracy measured their code according to ridiculous formal measures from the Central Bureaucracy and arbitrary rules that changed according political whims? That's what teachers are facing. I know some software devs face the same thing, but we always give them the same advice: find a better job. Teachers don't have that option because the bureaucracy is everywhere; but they can go to better (generally richer) schools, where it's a little mitigated, or they can drop out and leave that gaggle of little Kafkas behind forever.

If someone gives money, they want measurable, or at least visible, results.

If you want autonomy at the level L (federal/state/county/local), you can't take any money from any level greater than L.

You may not want complete autonomy, just a better national system. If that's the case, then you need to come up with a better way to measure results -- not inputs -- and suggest that. If it's really better, then people will listen, and maybe the existing standardized tests will be replaced with this better system.

There is less of a demand for testing at the local level, of course, because people generally know pretty well how their local schools are doing.

But nobody in Kansas is going to take a tour of schools in Alabama to see if the federal money is being spent wisely. They need a number. It doesn't matter how counterproductive you say it is to collect this number, they will only stop asking for a number when we stop asking for their money.

Yes, they do, don't they? I think this is the essential fact that keeps getting forgotten.

I've long supported voucher systems for just this reason: they return control to the parents. Maybe vouchers aren't the best way to do that; I'm open to alternatives. But I think that's what we have to do.

(Edit: formatting)

By all measures test scores are not improving in spite of quadrupling the DOE budget over the past 40 years. How about them metrics? This suggests we should abolish the pervasive bureaucracy.

Let me rephrase to be more precise: a person who votes for federal education spending will also tend to vote for federal control over education (regardless of the effectiveness of that control).

And that goes for everything. Federal spending means federal control over anything related to that spending. Anything else is sustainable only by wishful thinking.

$200 million goes to the Institute of Educational Sciences to track score data. In the absence of the IES would people pay this much for the data? I don't think so.

I'm not sure what you mean by "Anything else is sustainable only by wishful thinking." Education is not fundamentally different than any other good or service, besides that it is "important" and easy to convince the majority of people that it will cease to exist if the government doesn't provide it.

At least two candidates proposed eliminating the DOE during the Republican primaries. One was Rick Perry, where the DOE was one of the 2 departments he remembered to cut; and the other was Ron Paul, who may or may not have said it explicitly, but I'm sure he has said so at some point.

I disagree. People do vote for federal education spending. This pressure to spend comes from somewhere. However irresponsible it is, many people simply vote for more money, regardless of where it comes from. Some people will even vote to have money taken from them and given back -- strings attached.

"I'm not sure what you mean by 'Anything else is sustainable only by wishful thinking'"

I meant that it's wishful thinking to expect money to keep raining down with no strings attached. I agree that education is not fundamentally different from other goods and services.

I learned to fly people around in airplanes using various instructional techniques, including at times studying for the test. So do most airline pilots. Seems to work for them. The military has been training people for over 200 years with measurable teaching goals. In fact, some of the best teachers are ex-military. I could continue if you'd like.

"...if you're working anywhere worth working--there'll be a much higher non-formal bar set by your manager, your team, your customers if you're in contact with them..."

You are throwing out all requirements from any outside bureaucratic structure and then assuming that some informal standard of quality is going to fill the void. I don't know what your experiences are, but this kind of "let's see what happens" mentality doesn't trend toward good things. It's a tragedy of the commons problem. Why do a little bit extra here if somebody else in the organization is going to look at it anyway?

Instead, we find that really smart people are great at arguing on and on about technical stuff that has nothing to do with delivery. The only thing that naturally limits this is having testable work due on a fixed date. Otherwise, people would much rather create little complex systems that mirror their favorite technical book or management philosophy. There are no fights that are as vicious as academic fights over the smallest things.

The answer here is simple: if it's not measurable, we do not pay for it. I am sure there are lots of teams of really great people leading fulfilling lives creating quality in an ad-hoc manner that will continue doing so. We call these folks artists. There are also people who live the same kind of lives but have measurable results at the end of their creative process. These folks are usually called artistic professionals.

Teachers cannot be artists. The public cannot afford them to be so. They can certainly be artistic professionals, but like other artistic professionals there will be some kind of loose structure around their work product and it will be judged at regular intervals. This measure will be centered solely around students, which is natural because the only reason any teacher would be paid would be because of a student.

There might be a great argument about how the current tests are bad. But I think that's another discussion.

As a side note, and I want to make sure I hit this the right way, I do not care about teachers as such. Yes, I care about the dignity and humanity and kindness we show to all people, but teachers do not get some special place above, say, policemen. Or doctors. They're just like the rest of us. If you try to split up each profession and say we should treat them all in special ways due to the nature of their jobs, it gets much too complex for me to manage. Instead, when I meet somebody we pay our taxes to help us out, I usually try to say "thank you"

It doesn't matter to me whether we employ one teacher or ten million. If we could teach all of the students of the country with one teacher, that would be fine with me. The same goes for those other professions.

For instance, say you proposed creating music according to micro-goals. Say we live in the late 1800s. You look at the classical music and you come up with goals regarding tempo, intonation, meter, and some higher goals regarding fit and feel of the music. Then your Central Bureaucracy says, ok, this is the goal for creating music. Congratulations, you just killed jazz, and rock and roll to follow. All musical producers will have their acceptance criteria for recording a track, but it will be different from producer to producer, from musical style to musical style, more formal or less formal, and in general musical innovation comes from moving the goalposts or tearing them down. And yet these are the "creative professionals" you talk about.

In contrast, teachers are "artistic professionals" (except maybe at level of PhD advisers). Their goal is to use artistic techniques to create products which have achieved a fixed set of goals.

Bureaucratic goal setting is not desirable in all circumstances, but it is useful in some.

There is no central government agency counting how many lines of code per function point, how many spaces per tab, or what brace style I use.

Nor should there be. The people at the local level are the only ones qualified to determine how to test. And the success is proven by the fact that we are leaders in our segment, reviews are good, customers are happy, and we are profitable.

Government run schools do not have this feedback mechanism.

As a customer you're looking for a one-off administrative web application. In several years it's delivered (along with a warehouse of paperwork) in all of its 100% unit tested glory. As required by Federal Web Compliance 625, it uses EJB and JSP against an Oracle database. One of the tables has data duplication, requiring Federal Normalization Exemption 835.

It's so late because there was a brief controversy during development. One of the developers had increased his productivity by using a code generator to achieve 100% coverage. Suddenly he had a lot of time on his hands--he spent it refactoring the code, building prototypes, knocking out requirements, etc. Due to management oversight his transgression wasn't identified for several months, and the mysterious increase in productivity was hailed by local politicians as a shining example of how the system is working. Once identified, he was escorted out of the building, and a new developer had to rewrite his code from scratch. There was a brief splash in the local papers but that blew over, thank god...

During the meeting in which the resulting steaming pile of crap is delivered to you, the customer, one of the programmers breaks down crying. You can't put your arm around him and comfort him because it's against the law.

What are the problems with this? It seems rather fair right? A few problems: 1) The teacher in the previous year cheated on the test results (or the test changed), so that the students scored unnaturally high in the previous years 2) Gains may be unevenly distributed--is it as easy to move a student from 90% to 95% as it is to move a student from 20% to 40%? 3) Stability of the measure--if you look at this metric over say 5-10 years, is it consistent for a given teacher, or does it fluctuate? 4) If you run the metric backwards in time, is it still predictive? That is, if we look at this year's results, can we predict what the student would have achieved in the past? 5) What about background issues? This in some sense assumes that the background of student achievement is flat in the absence of a teacher--but middle class parents will spend a lot of time working with their kids, that poorer parents (on average) do not. This means that even if the teacher does nothing, the middle class students will likely still perform better. If these students are not equally distributed (which they won't be), this will also effect the metric.

I could go on, but my point is that this is one of the more objective methods of valuing student performance and it fails--I think there are several problems with schools--I think that professionalizing the teaching profession would go a long way in predominately middle class districts--for example, requiring teachers to obtain a masters, offering higher salaries to recruit more talented teachers into the field, requiring subject matter competency (my family has a friend who's background is teaching middle school math who got drafted into teaching geography! With relatively little time given in advance to prep.), etc. I think setting curriculum standards and tests that reflect them and allowing the teachers autonomy in how they teach the required subjects would be useful. Generally, promoting more understanding and less drill and kill--but just saying, "throw the whole thing out" is not terribly useful. There's a fun read called "Surpassing Shanghai" which offers a brief review of education systems in other countries that is worth reading...

What good is it to have measurable goals if you are measuring the wrong thing, or your measurements are corrupt, or there is no effective means of correction if goals are not met?

> To continue the programming analogy, as a customer you guys are delivering crap all the time. I pay ten times as much and the crap just gets worse. Now we can sit around and dicker all day long about the process you have for making crap, but one thing we are not going to do is remove the testing at the end that makes it obvious to everybody just how shitty the whole thing is.

Keep it in then. Your programming team can learn to game the system, and your management team can quietly lobby to change the metrics so the numbers look better, and you can even bring in a pair of Bobs for a development cycle to shake things up, but none of that actually gets you to working software.