I would recommend downloading the dataset[1] and taking a look at some of the essays. There is a tendency for an educated audience to (justifiably) worry about exactly what you've articulated, but when you read through the essays, you get an appreciation of how valuable a tool like this could be. These are elementary and middle school essays rife with rudimentary mistakes (which are at times extremely charming) and generally not filled with subtle arguments prone to misinterpretation.
If algorithms can help every seventh grader in the country write a solid 5 paragraph essay, that's progress.
I wrote about this more extensively here: http://bellm.org/blog/2012/04/23/arguing-to-the-algorithm-ma...
Two things: 1) Agree about the explanation bit. In fairness, that was not part of the competition. But I do believe that, given the way the features were constructed, many of the algorithms could be modified to also provide "explanation" of their scoring. The value is highly dependent on the features though (length, for instance, would probably be a prominent "explanation"), and I haven't actually tried it.
2) The "This would grade Dickinson/Hemingway/DFW/etc poorly" argument is true but, again, read the essays. These are serious edge cases - ones that should be addressed (by, for instance, correlating algorithmic scoring with human scoring), but this argument doesn't diminish the applicability of the algorithm to the vast majority of essays and students.
Edit: basically, the sheer scale of the grading required for a standardized test means that it's never going to be a real test of writing skills. It wouldn't surprise me if overly skilled writing is actually a negative even with the human grader—I could easily see a rushed grader misreading something clever and giving it a low score.
(The wisdom or effectiveness of these tests in the first place is a whole different discussion.)
I'd guess that the algorithm "map number of Microsoft Word reported spelling errors from (0 -> 800) to (20 -> 400)" does a pretty damn good job of guessing students' SAT Writing scores. Add in a factor for essay length, unique word cound, and grammar errors, and you'd probably get pretty damn close to perfect, within a reasonable statistical tolerance.
Children don't produce art, almost all of them produce garbage, right up to the top 1%. Make no mistake, all that standardized tests aim to measure is how stinky the shit that they produce is. I'm fine with missing a bit of brilliance at the top; those students will do just fine anyways. It's ranking the relative crappitude of the middle that's important here, and I think the standard (if crappy) grading procedures do a pretty good job here.
The best part about that is the fact that people misspell words even with spellcheck! Come on, how hard is it to notice the squiggly lines and correct your error by picking the correct spelling from the drop down list?
And I wouldn't say what I produce is shit, but then I would have said that when I was twelve. And I mostly have to double time on projects and essays, so they probably are shit.
EDIT: And since losethos (Schizophrenic dude who's hellbanned.) asked, the boss; it's pretty clever to use someones tendency for pedantry against them.
I say this as someone who graduated at the top of my high school class - if you can't get me to work hard for you, you're pretty screwed with the average student. It's no wonder that the stuff they wrote for you was crap.
What he said: "Most of these kids are straight-up idiots..."
He hasn't already decided, he just came up with a prior.
In CS, we often grade assignments automatically, although we expect students to write the assignments by hand. Although there are many classes where this would be a bad idea, there are some where it works quite well - if the problems are strictly specified, grading an assignment with what are effectively unit tests is reasonable.
Grading and doing an assignment are different classes of problems - roughly mapping to decision problems and search problems. Just because the grading is computationally tractable or acceptable does not necessarily imply that the act of writing the assignment is.
I agree with your second paragraph by and large. There is a time and place for automatic grading, and most writing should not fall into that category.
It's probably possible to grade student programs like this essay-grading system works: instead of feeding the programs to a compiler and running against test-cases, just turn the source code into a bag-of-words representation and feed it to a statistical classifier. There are almost certainly some statistical correlates between the source code of programs and whether they're good/bad submissions. But the results would be both less reliable and less informative as feedback.
Writing is an artform and as such should be treated and graded as one. Sometimes you need to use language that is grammatically incorrect to get a point across and software is not nuanced enough to pick up on this. I can see this helping to turn out students that are good grammaticians but terrible writers - optimizing for the wrong outcome.