News for hackers. Stuff that matters.
Re:The Algorithm Didn’t Like My Essay by _delirium on Sunday June 10, 2012 @11:34AM (#4091311)
One difference with CS algorithmic grading is that it's much more specific and causally connected to outcomes. It's not looking for statistical correlates of the input text and a notion of "good program", but is actually finding things like compile errors and failed testcases. And, crucially, it can also report those to the student: your program is wrong because when fed 5 it gives an output of 23 when the output should be 25.

It's probably possible to grade student programs like this essay-grading system works: instead of feeding the programs to a compiler and running against test-cases, just turn the source code into a bag-of-words representation and feed it to a statistical classifier. There are almost certainly some statistical correlates between the source code of programs and whether they're good/bad submissions. But the results would be both less reliable and less informative as feedback.

Re:The Algorithm Didn’t Like My Essay by jacktoole1 on Monday June 11, 2012 @12:55AM (#4093271)
Good point; I agree this makes code much more naturally machine graded than essays, and a statistical classifier for source code sounds disturbing (although it might do better on grading style than the usual autograding does).