I recall the (Texas) essay prompts used to be incredibly vague and had nothing to do with what we were learning in class at the time. Prompts like "Write about a time you overcame a challenge" come to mind. Back then, I assumed they were just testing our ability to communicate ideas to an audience, but really all they wanted was the default 5-paragraph essay. It was little more than fill-in-the-blanks to this outline, rather than actual writing.
What made the AP Literature and Rhetoric tests so good was that everything we learned in class was applicable to them. Obviously, my teachers took time to explicitly practice the AP essays, but every other activity also helped feed my knowledge of the subject that I could use to answer the essays on the AP test. Other standardized test essays just seem to be antithetical to any possible purpose.
One of the few perks of being a teacher is the extra vacation time compared to the rest of us. Cutting into that, with the typically already-poor wages, doesn't seem like a good plan.
http://news.ycombinator.com/item?id=1673075
Teachers also have many perks - summer vacation, minimal accountability, defined-benefit pensions, etc. In fact, a rather significant chunk of teacher's comp is perks, not pay, far more so than most people in the private sector.
(which is why I think "people sneaking into schools with fake residence addresses" should be punished; if they're trying to send their children to another district, it should be by making a sacrifice and working in the district, vs. indirectly taxing everyone who lives in that district)
Name any job other than "Congressman" in which a single nude photo of you will put you name in the tabloids and end your career.
And not only for grading, but for teaching as a whole. There is just no smart solution for educating children at a larger scale - we have to do "brute force". The less children per teacher, the better.
(Different story for adults who can autodidact, I guess, but even at universities the student / prof ratio is important).
One example of a teacher scaling is khan academy. I think that something like khan academy could have been done even before the internet, even with VCR's. And automated practice(in math) using computers was available when i was a kid and we even used it through grade 6 , but in grade 7 when we moved to another school we stopped using it.
I think the problem is more due to the fact that the teaching profession is conservative(mostly as a way to maintain power, which is not unique to teaching by any means).
And there's the money quote.
They are designed to be evaluated quickly, and objectively. Scantrons can be graded mechanically, and these essays described in the article can be graded with no thought and a minimum of judgment. These goals, efficiency and objectivity, impose constraints on how you can test competence, and as far as I can tell those constraints are simply insurmountable.
I have taken one and exactly one category of standardized test that I respected, and that was the AP tests. The essay questions for AP Lit was graded by having three English teachers read the essay and evaluate whether your answer indicated you understood the meaning of the passage, and how it relates to the work as a whole. In other words, it was done the hard way. And it was better than any other standardized test I've ever taken at correlating performance with understanding.
The problem is scale. Scaling is the only advantage of the current form of tests, and that's enough. Any replacement is going to have to address scaling.
(My hare-brained solution: Grading essays is what teachers do over summer vacation. It's enough excess labor to remove that design constraints.)