Re:AI Grading Application Gradescope Shortens Grading Times
by analog31 on Friday September 02, 2016 @11:34PM
(#12417582)
This makes a lot of sense, and follows typical methods that were done by hand long ago. When I TA'd freshman physics at a big university in the 90s, the grading session was a two-pass assembler. First pass, we simply identified the possible answers. Second pass, we marked them. This gave us much more consistency, and it was quicker overall.
Amusingly, the exams always asked for a numerical answer, from which we could guess which mistake they made, then we would find that mistake in their calculation and mark it. Without that trick, identifying the specific mistake in each answer was a pretty tedious process.
It seems like the AI is identifying equivalent answers among respondents. So if you mark an answer correct on one test, every other test with the same answer will be marked correct. I worked for a small competitor of this product in college and we had a lot of trouble with this problem, especially with answers that were prone to spelling mistakes, or could be written in many ways. Kudos to them for doing this well.
Another fun bit of AI in this space is in identifying where the answer key might have made a mistake. We developed some algorithms for determining the most likely answer to a problem given the responses. We never released it but I worked on a tool that would grade tests without an answer key at all. Using 50 question tests in a few freshman physics classes I was able to get the right answer a little over 97% of the time.