News for hackers. Stuff that matters.

A comment on GPT detectors are biased against non-native English writers. Parent comment

Re:GPT detectors are biased against non-native English writers by transcriptase on Sunday May 21, 2023 @10:30AM (#36021250)
I don’t disagree with you, though personally I’ve called out two people on Reddit using ChatGPT to farm karma by pretending to be an expert on some topic… and an hour later they deleted all of their GPT generated replies from their profiles.

There’s definitely a pattern for some prompts, where the model uses an obvious format of:

  - Make generalized statement answering prompt question
  - Support that statement with 3 or 4 discrete paragraphs that don’t exhibit any personal experience, give examples, or cite statistics
  - Finish off with a high-school essay style conclusion statement reiterating the introduction with different phrasing, generally beginning with “Overall” or “In conclusion”
It’s also comically obvious when suddenly someone who has a reply history full of low-effort juvenile, zero grammar, curse laden posts on video game and meme subs is suddenly writing mini-theses on topics ranging from Swedish forestry management to the chemistry of dyes used in Cambodian textile manufacturing.
[ Parent | on HN ]
Re:GPT detectors are biased against non-native English writers by ghaff on Sunday May 21, 2023 @11:27AM (#36021929)
There's definitely a "what are the benefits of" or "what are the reasons for" format that ChatGPT falls into pretty easily. It's usually not really wrong and would probably serve as a passable high school essay. (And I've probably seen worse marketing copy.)

But it's not very good and, at a minimum, lacks nuance and supporting evidence.

[ Parent | on HN ]
Re:GPT detectors are biased against non-native English writers by M4v3R on Sunday May 21, 2023 @01:01PM (#36022793)
The key word from GP is "reliably". Sure, you can probably tell when someone blatantly copied and pasted output from ChatGPT and when the prompt was not very sophisticated (write a comment about X). However for all other cases when someone put even the tiniest bit of effort into the prompt or did some post filtering the reliability of such detection plummets significantly.
[ Parent | on HN ]