News for hackers. Stuff that matters.
Re:The Algorithm Didn’t Like My Essay by gammarator on Saturday June 09, 2012 @11:36PM (#4090099)
I participated in the Kaggle competition, but I'm less optimistic about the pedagogic value of these systems. A black box algorithm won't explain to a student what their rudimentary mistakes are.

I wrote about this more extensively here: http://bellm.org/blog/2012/04/23/arguing-to-the-algorithm-ma...

Re:The Algorithm Didn’t Like My Essay by numlocked on Saturday June 09, 2012 @11:47PM (#4090115)
Great write up! I hadn't seen it, thanks for sharing.

Two things: 1) Agree about the explanation bit. In fairness, that was not part of the competition. But I do believe that, given the way the features were constructed, many of the algorithms could be modified to also provide "explanation" of their scoring. The value is highly dependent on the features though (length, for instance, would probably be a prominent "explanation"), and I haven't actually tried it.

2) The "This would grade Dickinson/Hemingway/DFW/etc poorly" argument is true but, again, read the essays. These are serious edge cases - ones that should be addressed (by, for instance, correlating algorithmic scoring with human scoring), but this argument doesn't diminish the applicability of the algorithm to the vast majority of essays and students.