There is a paper with more details "Five Years with Kattis - Using an Automated Assessment System in Teaching", https://www.csc.kth.se/~gkreitz/kattis-fie11/
The system is still developed and there is now an open version of it with a huge problem archive (competitive programming style) used by several universities: https://open.kattis.com/
Trivia: KATTIS was originally very much developed by KTH's successful competitive programming team "Three-headed monkey", consisting of some PhD students (https://www.csc.kth.se/~gkreitz/) that did their thesis on stuff like streaming and encryption which in turn resulted in the company... Spotify.
See: http://theory.stanford.edu/~aiken/publications/papers/sigmod...
However, if you changed the variable names, like in the article, the test cases would fail...unless you changed them in the test cases itself. In which case, the test cases would definitely fail in the online grader.
I took the Andrew Ng Coursera class which had an online grader that had test cases as well, but mostly it was just defining functions with specified input and outputs, so you're free to use whatever style guide you want in between. (CS100.1X does have a few of those, and they are easily the most difficult part of the assignment since it requires Python shenanigans)
For example, there aren't very many sane ways to write a Hello World program.
Anyone here have access to one of those code-plagiarizing detection systems used at CS schools? How would it fare on this kind of obfuscation?
I know that a plagiarized program that tries to just futz variable names is probably computationally easy to detect. And even extremely computationally exhaustive comparisons would be feasible, because the average college only has to deal with a few hundred to a thousand students a semester.
But would the algorithm even have to be more sophisticated than doing just a plain diff and applying some heuristic, even just the ratio of identical lines to total lines? My perception of students who would cheat in a 101CS class is that they are so incompetent that they don't even know how to change a variable name in a program without breaking the code...nevermind rearranging the order of a loop operation. Any student who could confidently obfuscate their plagiarism of 101-level program assignment is probably smart enough to just do the program themselves.