I wrote about this more extensively here: http://bellm.org/blog/2012/04/23/arguing-to-the-algorithm-ma...
Two things: 1) Agree about the explanation bit. In fairness, that was not part of the competition. But I do believe that, given the way the features were constructed, many of the algorithms could be modified to also provide "explanation" of their scoring. The value is highly dependent on the features though (length, for instance, would probably be a prominent "explanation"), and I haven't actually tried it.
2) The "This would grade Dickinson/Hemingway/DFW/etc poorly" argument is true but, again, read the essays. These are serious edge cases - ones that should be addressed (by, for instance, correlating algorithmic scoring with human scoring), but this argument doesn't diminish the applicability of the algorithm to the vast majority of essays and students.
I would recommend downloading the dataset[1] and taking a look at some of the essays. There is a tendency for an educated audience to (justifiably) worry about exactly what you've articulated, but when you read through the essays, you get an appreciation of how valuable a tool like this could be. These are elementary and middle school essays rife with rudimentary mistakes (which are at times extremely charming) and generally not filled with subtle arguments prone to misinterpretation.
If algorithms can help every seventh grader in the country write a solid 5 paragraph essay, that's progress.
[1]: http://www.kaggle.com/c/asap-aes/data