News for hackers. Stuff that matters.
Re:Signs of undeclared ChatGPT use in papers mounting by matthewdgreen on Friday October 06, 2023 @11:57AM (#37792458)
Four quick thoughts:

1. Bad grammar in a paper is incredibly irritating, which is particularly unfair for people who are non-native English speakers. If ChatGPT can help people to convert awkward English into nicer work without changing the ideas or confabulating, that seems like a win. I don't see why people need to "declare" their usage of ChatGPT for these purposes. Is that a standard now?

2. The major issue here is that the authors copied a response and accidentally also brought some button text with it. This is given to imply that people aren't reading ChatGPT's output. But I think this is just as likely to be benign. People do all sorts of silly nonsense in papers (including me), like leaving author comments in final drafts. It's bad and sloppy, but it isn't usually a crime.

3. Many papers on public servers are junky. Many journals and conferences are equally junky. Search for any bad string and you'll find it everywhere, and non-scientists will be scandalized.

4. I like the people at Retraction Watch and think they're doing the Lord's work. However sometimes I worry that immersing yourself in wickedness makes you see wickedness everywhere, and also creates an (unconscious) incentive to sensationalize bad stuff. I hope they're aware of this.

Re:Signs of undeclared ChatGPT use in papers mounting by not2b on Friday October 06, 2023 @12:34PM (#37792961)
The authors could be required to declare only to the reviewers that ChatGPT was used for the purpose of cleaning up the language. Reviewers could then be alert to check for artifacts like this, verify that no references were hallucinated, etc.

Perhaps a footnote would be added to mention that ChatGPT was used in this way.

Re:Signs of undeclared ChatGPT use in papers mounting by stayfrosty420 on Friday October 06, 2023 @11:59AM (#37792482)
how is (2) likely to be benign and not plagiarism? It is completely different from leaving an author comment in a draft...such a weird take.
Re:Signs of undeclared ChatGPT use in papers mounting by matthewdgreen on Friday October 06, 2023 @12:04PM (#37792542)
If I type a series of bullet points into ChatGPT and ask it to re-phrase them in better English prose, am I plagiarizing? If it's not plagiarism but I screw up at copy/pasting, does it become plagiarism? I don't know! It is not something I personally would ever do because I like my own writing. But I'm also not going to throw that accusation at ESL authors who are using a writing tool (like Autocorrect-on-steroids) to convey original ideas to an audience, at least not until we've had a much longer community discussion on using these tools.

ETA I would urge you not to throw the death penalty at people whose crime may be more akin to shoplifting. This is bad both for the shoplifters and for people who want to fight serious crime. Plagiarism is one of the most serious accusations in science: casually tossing it around like this risks our ability to enforce it against serious offenders.

Re:Signs of undeclared ChatGPT use in papers mounting by thfuran on Friday October 06, 2023 @01:20PM (#37793470)
>like Autocorrect-on-steroids

It's a difference in quality large enough to constitute a difference in kind. Autocorrect nominally takes text in your own words as typed and converts them into your words as intended. Chatgpt used as you describe does not produce text in the author's own words.

Re:Signs of undeclared ChatGPT use in papers mounting by matthewdgreen on Friday October 06, 2023 @01:28PM (#37793551)
As children in grade school we are taught a series of rules about plagiarism that are designed to make us extremely sensitive about what we copy from. We're taught this because if you let a group of children do whatever they want, they'll copy stuff straight off Wikipedia or each other and thus they won't learn how to write properly. So we teach them that writing every word yourself is the most important goal to aspire to.

As an adult in a scientific field, what actually matters to me is having original scientific ideas conveyed to me in an efficient manner. I'm not really interested in applying arbitrary elementary-school rules on authors, as long as they aren't stealing ideas and text from other people (thus misallocating credit in ways that are harmful to progress.) I think as a community we should proceed from these principles, then work the details out from there.

Re:Signs of undeclared ChatGPT use in papers mounting by thfuran on Friday October 06, 2023 @01:40PM (#37793696)
And it's fine if that's what matters to you, but I think there's legitimate room for contention on the meaning of authorship or plagiarism when it comes to something like this. If a work is written in one language and translated by another person into English, I'd expect the translator to be credited. Is the sole purpose of crediting the translator acknowledging their work or does it also serve to acknowledge that the other author can't necessarily vouch for translation? If it's not translated by a human, should the translation software be similarly credited? If the writing process consisted of one person writing a bulleted list outline and another person converting that to full prose, I'd expect that person to be credited as an author. What about the case where that step was done by software?
Re:Signs of undeclared ChatGPT use in papers mounting by matthewdgreen on Friday October 06, 2023 @01:47PM (#37793768)
My point is that it's not what matters to me. Our rules shouldn't be vacuous, they should be founded in real goals that advance science. In science we want people to communicate ideas clearly, we want the ideas to be accurate, and we want to allocate credit properly to authors. Adding a rule that says "you must say you used ChatGPT or you get punished" is unlikely to advance any real goal -- it will just lead to most authors adding that footnote, and a few people getting arbitrarily punished for forgetting it.
Re:Signs of undeclared ChatGPT use in papers mounting by thfuran on Friday October 06, 2023 @02:32PM (#37794428)
>Our rules shouldn't be vacuous

That something concerns matters of little personal interest to you doesn't make it vacuous.

Re:Signs of undeclared ChatGPT use in papers mounting by dragonwriter on Friday October 06, 2023 @01:35PM (#37793627)
> As an adult in a scientific field, what actually matters to me is having original scientific ideas conveyed to me in an efficient manner. I'm not really interested in applying arbitrary elementary-school rules on authors, as long as they aren't stealing ideas and text from other people.

The elementary school rules, at least the ones I weas taught decades ago, are pretty much “don’t steal ideas and text from other people”. Like, literally, other than for formatting, there were exactly three rules avout plagiarism I was taught on this in elementary school, and only one of them becomes slightly fuzzier outside of that context:

(1) if you use exactly wording from someone else, put it in quotes and cite the source,

(2) if you use an idea from someone else, cite the source but don't ise quotes, and

(3) if an idea comes from three or more independent sources, you don’t need to cite it, it can be treated as “common knowledge”.

Outside of elementary school, yeah, whether something is genuinely common knowledge or should be cited to multiple sources is fuzzier and context dependent, but the other two rules apply pretty directly.

Re:Signs of undeclared ChatGPT use in papers mounting by matthewdgreen on Friday October 06, 2023 @01:40PM (#37793695)
If the source is Microsoft Word's grammar correction engine, do we require people to quote and cite that? The question here is whether "quote and cite" should apply to ChatGPT (used exclusively for re-wording existing text.) And I think my principles in the post above provide a better framework for thinking about that question than the grade-school rules (and rules for citing texts, which is again a different situation.)
Re:Signs of undeclared ChatGPT use in papers mounting by thfuran on Friday October 06, 2023 @01:42PM (#37793724)
>“don’t steal ideas and text from other people”

Then the rules you were taught were far more lax than those I learned. According to the rules I was taught, copying lines from your own prior work without attribution would also be plagiarism.

Re:Signs of undeclared ChatGPT use in papers mounting by folmar on Friday October 06, 2023 @02:17PM (#37794197)
> If I type a series of bullet points into ChatGPT and ask it to re-phrase them in better English prose, am I plagiarizing?

I think the OP's point is that you could be, and should check, as such models sometimes reproduce parts their training set. You should check what you send, or, in other words, take responsibility for the tools you use -- the tools that are bread and butter are somewhat excused, but LLM are not at the moment.

Re:Signs of undeclared ChatGPT use in papers mounting by JohnFen on Friday October 06, 2023 @01:00PM (#37793224)
I don't know. The use of an LLM to help produce something isn't necessarily a bad thing -- but in my mind, it does reduce my trust in what the thing is saying. It means that if I actually care about what the paper (or whatever) is talking about, I need to spend a lot more time confirming what it's saying. If I don't care much about what the paper is talking about, I'm more likely just to discount the paper and move on.
Re:Signs of undeclared ChatGPT use in papers mounting by StableAlkyne on Friday October 06, 2023 @01:23PM (#37793502)
The challenge here is if their English skills are so poor that they need a ML model to help make it readable, their English skills are likely so poor that they will not spot grammatical changes that adjust the meaning of the work. That's a problem when the next person finds that paper on Google Scholar and cites it - you get a telephone game in the literature. It's hard to remove bad "facts" from the literature after a few people cite it, since you also need to go after a whole chain of papers, hoping their journals are willing to issue a correction. God help you if it's in the intro section and the authors complain because of academia's "corrections are bad" mindset. Anything that increases the BS rate of the literature is a bad thing.

Also, if you go to contact the author or view a talk, and suddenly their English is broken and completely different than what they wrote, you start questioning whether they wrote it or paid someone to without disclosing it.

IMO it's worth disclosing for many of the same reasons you disclose funding, ethics approval, author contributions, and conflicts of interest at the end of an article: it's metadata that protects your credibility as an author by showing that you're not doing anything underhanded.

Re:Signs of undeclared ChatGPT use in papers mounting by breakwaterlabs on Friday October 06, 2023 @01:35PM (#37793631)
The article also gives the example of the phrase "as an AI language model, I..." being found in published papers.

The fact that this slipped by the paper's author, their editor, *and the journal review* is not benign.

I agree with the author here: is no one reading these papers before putting their names behind them? Is the entire journal system entirely fraudulent?

Re:Signs of undeclared ChatGPT use in papers mounting by matthewdgreen on Friday October 06, 2023 @01:36PM (#37793648)
See point (3) in my post above.
Re:Signs of undeclared ChatGPT use in papers mounting by StableAlkyne on Friday October 06, 2023 @02:05PM (#37794019)
Peer review is a mixed bag. It's the best system we've come up with, but still can be hit or miss.

Depends on how much the journal is willing to push the authors (predatory journals don't give a heck, prestigious journals usually care more because they've got a rep to uphold). Reviewers can be anyone from an expert in the field, a competitor who gives you a hard time (usually they're not actively malicious though), to some random prof doing it because the editor couldn't find anyone else, or even a grad student in the lab of a prof (it's a common training exercise).

I've had articles with anywhere from 1-3 reviewers, there is no standard beyond "the editor is a basic crackpottery filter, there is an external reviewer, and a copy editor makes it fit for viewing"

Re:Signs of undeclared ChatGPT use in papers mounting by breakwaterlabs on Tuesday October 10, 2023 @11:46AM (#37833643)
If the system is not catching "I am a language model", I have zero confidence in its ability to detect crackpottery, much less more insidious things like P-hacking.