Re:Scientists claim >99 percent identification rate of ChatG...
by vidarh on Thursday June 08, 2023 @10:20AM
(#36242305)
me to chatgpt:
> Here is some text. Please rewrite it as follows: 1. DO NOT change the content more than the bare minimum needed to comply with the instructions. 2. Rewrite to add at least one question mark, and one semicolon. Add a "however", "but", "although" and "because", and rewrite the sentences so it flows naturally and without changing the meaning. Here is the paragraph you should rewrite:
>
> They believe their classifier is effective, because it homes in on a range of stylistic differences between human and AI writing. Scientists are more likely to have a richer vocabulary and write longer paragraphs containing more diverse words than machines. They also use punctuation like question marks, brackets, semicolons more frequently than ChatGPT, except for speech marks used for quotations.
ChatGPT output (GPT4):
> Although they have confidence in their classifier's effectiveness due to its ability to identify a variety of stylistic differences between human and AI writing, a question emerges; why is this believed to be the case? It's because scientists are generally more likely to utilize a richer vocabulary and construct longer paragraphs filled with a diverse set of words, but can this solely account for the difference?
>
> They also employ punctuation, such as question marks, brackets, and semicolons, with a higher frequency compared to ChatGPT; however, an exception is the use of speech marks for quotations.
Of course their detector might well handle that too, but GPT4 is perfectly capable of writing text with very varying level of styles and vocabulary, and if their detection relies on that, odds are people will prompt their way around that quite easily before even looking at more advanced approaches like e.g. finetuning on scientific papers within the right field etc.
> Here is some text. Please rewrite it as follows: 1. DO NOT change the content more than the bare minimum needed to comply with the instructions. 2. Rewrite to add at least one question mark, and one semicolon. Add a "however", "but", "although" and "because", and rewrite the sentences so it flows naturally and without changing the meaning. Here is the paragraph you should rewrite:
>
> They believe their classifier is effective, because it homes in on a range of stylistic differences between human and AI writing. Scientists are more likely to have a richer vocabulary and write longer paragraphs containing more diverse words than machines. They also use punctuation like question marks, brackets, semicolons more frequently than ChatGPT, except for speech marks used for quotations.
ChatGPT output (GPT4):
> Although they have confidence in their classifier's effectiveness due to its ability to identify a variety of stylistic differences between human and AI writing, a question emerges; why is this believed to be the case? It's because scientists are generally more likely to utilize a richer vocabulary and construct longer paragraphs filled with a diverse set of words, but can this solely account for the difference?
>
> They also employ punctuation, such as question marks, brackets, and semicolons, with a higher frequency compared to ChatGPT; however, an exception is the use of speech marks for quotations.
Of course their detector might well handle that too, but GPT4 is perfectly capable of writing text with very varying level of styles and vocabulary, and if their detection relies on that, odds are people will prompt their way around that quite easily before even looking at more advanced approaches like e.g. finetuning on scientific papers within the right field etc.