Scientists claim >99 percent identification rate of ChatGPT content

Re:Scientists claim >99 percent identification rate of ChatG... by ironSkillet on Thursday June 08, 2023 @09:45AM (#36241792)

I think what will happen is that there effectively be an arms race between the bot generation and detection sides. As anomalies are discovered in the distribution of content generated by bots vs humans, bots will update to remove the anomaly. And researchers will try to find new anomalies. In the end as bots get better, the difference in the two distributions will be indistinguishable in practice.

[ Parent | on HN ]

Re:Scientists claim >99 percent identification rate of ChatG... by marginalia_nu on Thursday June 08, 2023 @09:49AM (#36241872)

Possible, sure. Likely, maybe. But you made it sound inevitable? Of that I am less sure.

[ Parent | on HN ]

Re:Scientists claim >99 percent identification rate of ChatG... by vidarh on Thursday June 08, 2023 @10:05AM (#36242076)

For any accessible detector you can automate the evasion. E.g, crudest variant would be:

   generate text
   run it through the detector
   finetune model w/result of the detector
   repeat until detector fails to detect.

But every time the failure rate of the detector starts getting high, the other side has a lot of development work to do finding new patterns that won't also lead to lots of false positives.

So while getting a point where it's totally impossible to detect may well take a long time, it'll be periods of the detector suddenly being hard to bypass for a while, and gradually getting easier, repeat. It's highly unlikely it'll ever be impossible to bypass for any length of time.

The only way of keeping such a detector working for any length of time will be to make it expensive or very hard for adversaries to get access to, which means effectively relying on only giving access to trusted users.

[ Parent | on HN ]

Re:Scientists claim >99 percent identification rate of ChatG... by ulucs on Thursday June 08, 2023 @10:28AM (#36242417)

Isn't this basically how autoencoder models are trained?

[ Parent | on HN ]

Re:Scientists claim >99 percent identification rate of ChatG... by marginalia_nu on Thursday June 08, 2023 @11:21AM (#36243395)

This assumes the ability to perfectly model this function, which is in a sense what we're setting out to demonstrate. It can certainly not be taken for granted.

[ Parent | on HN ]

Re:Scientists claim >99 percent identification rate of ChatG... by vidarh on Friday June 09, 2023 @03:01AM (#36254305)

Firstly, it doesn't need to perfectly model it. It needs to model it to within a range where claiming to have detected it causes more false positives than acceptable, and given the range of quality of human output that's a pretty wide span.

But secondly, LLMs wired up with a feedback loop and and appropriate IO can model Markov models (models, not chains) which are Turing complete, so a LLM wired up this way is clearly Turing complete as well. As such it's a question of training, not whether or not the model can model it.

[ Parent | on HN ]