> I imagine getting into the meat of the paper would quickly reveal issues

This is a tautology: the thing that can be validated can be validated.

I think it has an important difference for ChatGPT - it likes to generate numbers that make absolutely no sense. A human that lies will try to generate sufficiently correct data to convince. ChatGPT often won't even make an attempt to produce values that fit.

I guess that's good for us meat beings. Better for an AI to incompetently lie than competently lie.

I wonder if having AI models available would make it significantly easier to identify material created by that model. Seems it would be easier, but it would still be a big problem. Nets can be trained to identify ai vs not ai on a given model. And that's without needing access to the model weights, just training examples. But when there are N potential models...

Edit: Second paragraph added later.

ChatGPT right now also likes to make up fake citations when you ask it how it knows something, which could be checked quickly.

...request an example [I've tried, avail is none].

>they correctly identified only 68% of the generated abstracts and 86% of the genuine abstracts.

I think you and I are basically in alignment... what this tells me is that 14% of real abstracts are so bad that other human beings call their BS. Meanwhile, this AI stuff is kinda working 32% of the time in generating legitimately interesting ideas.

So at that point - yeah, that sounds about right. The 32% is still so low that it shows AI is not anywhere near maturity, whereas 14% of human-generated is crap.

And, yeah - a short blurb like an abstract seems to be exactly the kind of text that ChatGPT is conditioned to do well generating. As others below note - once a human starts reading the rest, the alarm bells trigger.

(Buuuut let us also enjoy the fact that ML can generate a decent-looking abstract to a scientific paper 32% of the time. This represents massive progress; I would venture that most humans cannot write such an abstract convincingly.)

I would argue the opposite. Give anyone who can write reasonnably the abstract of a few dozen papers in a given field, and I am confident they would be able to produce a convincing bullshit abstract. Most abstracts in a given field sound extremely similar, and are mostly keyword dropping with an ounce of self-promotion. I actually think that the only reason ChatGPT did not produce a higher rate of convincing abstracts is that the reaserchers assessing them knew they had a high probability to look at an AI result and were thus extra careful. Most of the abstracts correctly labelled as AI would probably be considered legit (which does not mean good) if sent to a conference without warning.

It really would depend on the quality of the work/paper. I bet mis-identification by humans of hand written or generated abstracts goes down in higher tier conferences vs. lower tier ones, since the writing standards in lower tier conferences can be pretty bad (well, in CS at least).

I suspect you are right Sean - and I know you are a guy who has read a lot of papers and been to plenty of conferences.

>Meanwhile, this AI stuff is kinda working 32% of the time in generating legitimately interesting ideas.

I don't know that interesting has anything to do with whether people found them plausible.

> ...this AI stuff is kinda working 32% of the time in generating legitimately interesting ideas.

The bar is not "this is legitimately interesting," it's "I've seen real abstracts worse than this."

What should result is a more fortified peer review system.

All true I think. Interesting thing is how many researchers read just the abstract of papers. I’d say it’s quite common to read just the abstract and some future captions, and only dig into details on a few papers.

That's the main purpose of the abstract. There are several levels of reading a paper: checking the title, reading the abstract, reading intro/conclusions and skimming over the table, or reading the whole thing. You start at the first level and continue or stop depending on the interest of the paper for your work (and available time).

This works very well to more or less stay on top of things with today's frantic publication pace in many disciplines.