I'll take your point further. If you have a college class of 1000 students and they submit 5 papers a year, that is 5000 papers. 1% miss would be 50 student assignmentss accused of plagiarism. This isnt detecting cats, this is about lives being seriously affected.

Further, i'm curious how universal the accuracy is...do certain subsets incur more of the burden?

The simple fix is don't determine grade based on coursework (because cheating is too easy), instead use exams.

The best part is that this is already how grades are weighted at any accredited school.

Very much education level and field dependent. It's certainly the case that high schools will grade on coursework more. And, further, many fields primarily grade based on papers, because these are the currency of knowledge, and research skills are difficult to test in an exam setting.

> research skills are difficult to test in an exam setting

I understand you're not saying it's impossible, but is it really that much of a pain for a real human to just read the paper?

Is not even that - this entire idea just misses the point entirely.

It doesn't matter if a very particular model $A$ under particular conditions can be identified with another particular model $C$. That can very easily be circumvented by using some other model (Vicuna, GPT-4, Claude, etc) in different conditions to fool the model $C$. It's a constant race that will never be won and shouldn't even really be attempted.

One idea for determining if the content is generated by OpenAI that requires far less work is to for OpenAI to just make a database table of (document_id_hash, text_shingle_hash, shingle_weight_by_frequency) to make a system like old plagiarism detection. It's incredibly fast and simple, so uses far less compute than a big ML model to do an approximate but worse method.

Yes, yes, the API, other models, etc. But that was already a problem (see above).