How practical is this?

1. Is your institution able to provide this support?

2. Do you believe you are able to supervise the room well enough that students will be caught if they cheat? (Eg, bring a phone and look up answers.)

I don’t know about 1s practicality. In my schooling it would always be doable but I have the impression US schools are a different scale

If you can escape the vm on the fly and manage to use an llm to cheat, you deserve that A.

This is basically how ICPC works (https://icpc.global/)

Contestant computers boot a Linux image that contains all the interpreters/compilers and documentation for the supported languages, some editors/IDEs(vim, Jetbrains, Eclipse, etc) and a web browser.

Network access is limited to the judging system only.

My experience in Europe is that most universities can support this.

If you're doing a good job, you have to do that anyway, or at least have enough of a spidey sense for broken code to know when to investigate and add an extra test case.

Something like 30% of the time at $WORK, interviewers report the candidate as having solved the problem when a closer inspection reveals UB, memory corruption, and other bullshit. The test cases pass, and I think that's part of the problem. You can't tune out and avoid deeply understanding the submission.

I think the problem is that the grader has to run your code in their head. That's a whole different problem.

I'd say compared to just typing the code on a disconnected computer: Not worth it.

The grader has to grade N exams.

How do you think English, philosophy, or history tests get graded?

Or were you just assuming your experiences (large university, I'm guessing) are universal?

There are many, many colleges that have no grad students. Period. They are undergraduate colleges.

Expecting any college or university to just be able to scale up the amount of manpower required to grade their classes/exams by a large multiple is absurdly unrealistic. I assure you that most existing colleges do have plenty of resources to effectively evaluate individual students' outputs, if they don't have to assume that anything submitted digitally is written by LLMs.

That new requirement is a massive change to how they operate, and it's incredibly rude and short-sighted to be dismissive and contemptuous of institutions that have trouble adjusting to it.

That Computer Science is somehow a privileged subject where the exam grading must be automated is elitism of the highest degree.

So in some theoretical hiring example, if I have to choose between two candidates: one which is really good at keeping the code in the head and the other who is less good at that, but who is very skilled at using programming tools like advanced type systems or formal verification tools, then I'd prefer the latter.

Of course in practice, I would of course take a student that I knew had passed their paper test with their own knowledge over a student that likely ran their digital test through an LLM.

> or whatever is the tech of the day.

Rote memorisation has been the standard technique probably for way longer that anything else so I’m not sure that every innovation is inherently inferior to what we had before

No wonder some people are so easy to replace with an LLM.

this would also end up valorizing particular ways of thinking or developing, so those whose development methodology is more exploratory until you get a certain point where you get an overarching view of how everything should interact and then go into a quicker iterative mode would be penalized.

I guess it's ok it does this, some things after all must be valorized whatever is chosen, although I've known many people who are quite good using this way of working who would then be downgraded.

Interesting. This reminds me of COBOL and FORTRAN "code sheets" that were used back in the day...

We had 3 hours to make the test. The papers the teacher delivered had differences between them (I think it was four different versions of the test?). The Teacher didn’t stay in the class also, you had to deliver the test to them later in their office.

IME, that kind of nitpicky grading mostly happens when the exam was designed too easy and does not depend on the medium.

> Remember, the hand written code is also harder for reviewers to grade! You have to manually run the code in your head, for example

Which you really need to do anyway because code that is broken due to a brainfart and code that is broken because of a fundamental misunderstanding should not receive the same grade.