News for hackers. Stuff that matters.
Re:GPT detectors are biased against non-native English writers by AnthonyMouse on Sunday May 21, 2023 @12:15PM (#36022383)
> Are getting good marks that much more valuable than the actual education you would receive if you didn’t cheat?

The problem here is the incentive structure. The nature of grades is that some people will get As and some people will get Cs. If you get Cs, you might like to get As, but maybe you can't for some reason, from lack of intelligence to other time commitments to alcoholism.

Then your choice is to get a C on your own or maybe get a B+ by cheating. Given the same amount of learning, getting a better grade is better, so the only real incentive not to do it is the risk of getting caught. Some of the old methods for this were pretty unambiguous. If you submit the exact same essay as another student, or one that has been on the internet for ten years, what other explanation is there?

Rich kids would avoid this by paying someone else to do it for them. That has always been a problem. But now the cost of getting "someone else" to do it is approaching zero when the someone else is AI, so the problem spreads. But attempting to detect it with methods that have false positives is worthless.

On the other hand, testing students on their ability to do something that ChatGPT can do just as well? Maybe that's worthless too, because that's no longer a marketable skill when your future boss can get ChatGPT to do it too. So what they need to do is change the test to test for the thing the student is expected to be able to do better than the AI.

Re:GPT detectors are biased against non-native English writers by andai on Sunday May 21, 2023 @03:33PM (#36024044)
>what they need to do is change the test to test for the thing the student is expected to be able to do better than the AI

Do such expectations still exist? Are they expected to remain for very long?

Re:GPT detectors are biased against non-native English writers by AnthonyMouse on Monday May 22, 2023 @12:38PM (#36033620)
Clearly they do.

Have you taken some time to play with these things? Try the biggest LLaMA model that will fit in RAM on your computer. (128GB of DDR4 is around $200 and will just fit the big one without quantizing, though it won't be super fast.)

There are things they're good at. Search engine-like tasks in particular, if you're willing verify the output. They're great at providing hints for further reading.

Now try to get it to develop a new kind of battery with a longer service lifetime or lower manufacturing cost per unit energy storage. Ask it to write code to do something complex and uncommon instead of something similar to what it was trained on a thousand examples of. Have it describe a new class of security concern, like Spectre or rainbow tables before they were known.

People can do those things, and have done, and those are some of the best things we can teach people how to do because they're incredibly useful. Maybe writing minor variants on common existing boilerplate code isn't something we need people to spend a lot of time on anymore, and so isn't the thing we should be testing if they know how to do.