In that case, I hope these frauds have been banned for life.

I was thinking this too, but I don't believe this is the case, and I feel like it would not be a good idea either.

Most of these people are likely students; this should be a learning moment, but I don't think it is yet grounds for their entire academic career to be crippled by being unable to publish in a top-tier ML venue.

If this is tolerated, it sends exactly the wrong kind of message. The students, if they are, should be banned for life. Let them serve as an example for myriads of future students, this will be a better outcome in the long run.

This didn't trip for people who were merely bouncing ideas off a LLM, they caught people who copy and pasted straight from their LLM.

It's not a fully consensus view, but a majority of sociologists agree that high severity deterrence has limited effectiveness against crime. Instead, certainty of enforcement is the most salient factor.

Deterrence is only part of it. It's morally instructive, it tells people that they live in a society that takes rules seriously.

What is the aim of "moral instruction" if not deterrence? Surely it needs be instruction in pursuit of an outcome?

It makes honest people feel rewarded, valued and acknowledge. It teaches people who wish to follow the rules and conform to social norms what those norms are and where we actually draw the line in practice.

https://en.wikipedia.org/wiki/Punishment#Education_and_denun...

Looked at slightly differently, given a split between high trust and low trust preventing conversions from high to low is similarly important to inducing conversions from low to high.

Enforcement without consequences just wears down the people who are supposed to enforce it.

There's a pretty large area between "no consequences" and "banned forever"

GP suggested a life ban. Maybe suspend for 6 months instead? That's a long time without publishing in the current publish-or-perish academia.

> Maybe suspend for 6 months instead?

Suspend for 6 months from a conference that is held yearly?

I wasn't thinking about ICML specifically. My mind was on the ARR.

Correct. We also have evidence both from cheating in sports and in academia that stiff punishments do not work. Many people hold the false belief that if it is easy to cheat then the punishments must be extremely severe to scare would be cheaters. It just does not work. Preventing cheating is way easier said than done.

> We also have evidence both from cheating in sports and in academia that stiff punishments do not work.

Maybe so, but there is evidence that lack of punishment also don't work.

Neither extreme "works". Just because terminal punishments do not prevent the worst cheating does not in any way imply that slap on the wrists reduce incidents of cheating.

That's the American spirit. "We got to do something!" "Does it work?" "We must do something!"

I don't understand what point you are making.

Are you claiming that one of the extremes "works"? That the "light punishment" route reduces cheating? Or maybe has no effect on cheating?

There are two extremes; I am not arguing that the one extreme (terminal punishment) reduces cheating, I am saying that the other extreme (light punishment) does not reduce cheating!

You say that stiff punishments have no effect on the cheating rate, right? Compare to what exactly? Compared to no punishments? Compared to light punishments? Compared to medium punishments? Compared to heavy but non-terminal punishments?

Now that I've reread your comment, I'm extremely skeptical that terminal punishments have no effect on the cheating rate compared to light punishments or compared to medium punishments.

It's an extraordinary claim, so I want to see this "lots of evidence"; the evidence should basically show no correlation between cheating and punishments.

I want to see that chart you base your belief on.

But this method is now spent, as if someone is determined on keep using LLM, this should be pretty easy to overcome.

I suppose though new methods could be devised, but it's not "certainty" that they will catch them.

That's not true. People still pick up USB sticks from the street, people still fall for scam phone calls and people still click on links in mail.

Just because a method was successful once does not mean it was 'burned', none of these people will be checking each and every future pdf or passing it through a cleaner before they will do the same thing all over again and others are going to be 'virgin' and won't even be warned because this is not going to be widely distributed in spite of us discussing it here.

If anything you can take this as proof that this method is more or less guaranteed to work.

But the mob wants their kick.

Yup, precisely this. Doing something bad is rarely a rational commitment and cost of benefits. Likelihood and celerity of getting caught seem to be the driving factors.

The point of a punishment is not solely to deter future crimes, it's also to actually punish the present crime though

For instance jail time is not *just a deterrence, it's physically preventing someone from committing more crimes against the public

> Instead, certainty of enforcement is the most salient factor.

hodgehog11 is proposing effectively no enforcement

It has been shown time and again that, for most people, teaching them to be better and giving second chances is more effective than using forever-punishment as a warning for others.

Why not put them on a chain and let village stone them? Or better yet shoot them on the spot! That would send a message for sure.

> The students, if they are, should be banned for life.

I'm all for repurcussions ... but a life is a long time and students are usually only at the beginning of it.

My understanding is that something among those lines happened:

> All Policy A (no LLMs) reviews that were detected to be LLM generated were removed from the system. If more than half of the reviews submitted by a Policy A reviewer were detected to be LLM generated, then all of their reviews were deleted, and the reviewer themselves was removed from the reviewer pool.

Half is a bit lenient in my view, but I suppose they wanted to avoid even a single false positive.

This line of reasoning interests me because it seems to arise in other contexts as well.

Do very harsh punishments significantly reduce future occurrences of the offense in question?

I've heard opponents of the death penalty argue that it's generallynot the case. E.g., because often the criminals aren't reasoning in terms that factor in the death penalty.

On the other hand (and perhaps I'm misinformed), I've heard that some countries with death penalties for drug dealers have genuinely fewer problems with drug addiction. Lower, I assume, than the numbers you'd get from simply executing every user.

So I'm curious where the truth lies.

Is the death penalty scarier than life in prison?

I assume that depends on the individual.

But FWIW, my point was about very harsh punishments in general, not specifically the death penalty.

I'm not sure it was meant that way, but nice metaphor. For some students "academic death" might really be better than a life of being trapped in a system that they can only navigate by cheating.

agree

FYI we tend to use up votes rather than "I agree" comments, partly because it keeps the overall signal-to-noise ratio for comments higher.

Thank goodness we have you passing judgment on the internet; otherwise who else would be around for us to do it? I'm glad you're willing to destroy someone for a mistake rather than letting them learn and change. We all know that arbitrary and harsh punishments solve everything.

> destroy someone for a mistake

"Oops, you told me not to do this, and I volunteered to agree to these stricter standards yet I flagrantly disregarded them, please forgive me" doesn't seem like something you just accidentally do, it's a conscious choice.

Well, maybe they found themselves in the last hours of the deadline without the reviews done... in some cases due to procrastination, but in a few cases perhaps because life is hard and they just couldn't do it. So they used the LLM as a last resort to not go beyond deadline (which I assume maybe was penalized as well?)

To err is human, it makes sense that they are punished (and the harshest part of the punishment is not having a paper rejected, it's the loss of face with coauthors and others, BTW. Face is important in academia) but "for life" is way too much IMO.

This year, having their own submissions desk-rejected is strong enough of a signal that the policy has some teeth behind it. Let’s ban em for life next year.

I strongly feel that deterrence should be the goal here, not retribution IMO.

Between banning someone for life and not doing anything, there usually are some other options.

Like burned at the stake, tarred and feathered, drawn and quartered, etc.?

- return to drawn and quartered in the town square?

ML reviewing is a total joke. Why do you have noob students reviewing a conference paper.

Because someone has to do it. Conference submissions have ballooned as the field itself has ballooned.

Whats your suggestion?

It's better if nobody does it than to send it to the randomizer.

Ok but you need peer reviewed publications to graduate with a PhD.

And if you retort that the whole academic system is obsolete, well, it still carries a lot of prestige and legitimacy that makes politicians interested in maintaining it, so it's not going anywhere soon.

It's usually not "noob" students. Big conferences require reviewers to have at least one (usually more) published paper in major venues. For students, this usually means they went through the process of being the first author on a few papers.

I've been an AC (the person who manages the reviewing process and translates reviews into accept/reject decisions) at ICML and similar conferences a few times. In my experience, grad students tend to be pretty good reviewers. They have more time, they are less jaded, and they are keener to do a good job. Senior people are more likely to have the deep and broad field knowledge to accurately place a paper's value, but they are also more likely to write a short shallow review and move on. I think the worst reviews I've seen have been from senior people.

2% would be on the very low end of the number of people who lie, get caught, and become repeat offenders anyway.

What makes you think these are mostly students? I may have missed that in the methodology

In many cases authors and reviewers are not the same. In your first two publications to such venues you are not allowed to review yourself and need someone else.

I think consequences are well deserved, but hopefully not on the authors cost (if innocent).

What terrible deeds have you done to outburst so harshly?

It’s an unethical, false choice. The reviewers are not perfectly rational agents that do free work, they have real needs and desires. Shame on ICML for exploiting their desperation.

Banned for life is a stretch but the actual response is completely fine. They can just resubmit to the next conference.

Words mean something, if you promise to uphold a contract and break it, there are consequences. The reviewers were free to select the policy which allows LLM use.

Is it? The reviewers could simply have chosen a different option in a form field. While I understand that they were "forced" to review under reciprocal review, they still had other choices where I don't see coercion happening and that could have avoided the outcome for them.

I'm not sure what experience anyone in this thread has with grad level research as a student/author, but I can assure you that heads roll over this kind of thing.

A professor's career is built on reputation, and that reputation is as strong as their students' (who do much of the "work" such as it is). It comes down to the professor, but this can be a career-ending moment for those students and I'm quite confident there were some very uncomfortable discussions as a result of this.

It's just a tool.

Writing papers is exhausting, and if the data and results are real, then what's the problem? If the human author checked the output, is that not the same as a human writing the prose?

Everyone in the field will be doing this in a few years anyway. It's a shame that this Salem Witch Trial is happening for the early adopters.

If the findings are being fabricated or the paper isn't being reviewed and corrected by the author, that's a different story. But I'd be shocked if that were the case.

I consider LLMs to be a very useful tool and use them every day. But if I sign a slip of paper saying I won't use them for some project, and then use them anyway, not merely using them but copying without even the pretense of putting it into my own words, then that's fraud. LLMs being a tool is completely orthogonal to this fraud.

This comment doesn't seem to fit the discussion at all?

The discussion is not about humans using LLLs to write papers. It is about humans who agreed not to use LLVM in reviewing papers, then did exactly that.

There's a lot of irony in a defensive comment being written based on misreading / inattentive reading of a post about reviewing papers (requiring attentive reading).

In addition to being a reviewer, they also submitted their own research. So it leads to the question: if they were willing to cheat on the side of review with less incentive, why wouldn’t they cheat on the side that provides more incentives?

(Meaning, your career doesn’t get boosted much for reviewing papers, but much more so for publishing papers)

It might be that paper authors required others not to use LLMs for reviewing their work. Then, by the rule of reciprocity, they shouldn't use LLMs for reviewing others work. The article is unclear on whether this implied reciprocity rule was explicitly stated or not.

It was. More details here: https://icml.cc/Conferences/2026/LLM-Policy

In particular: "Any reviewer who is an author on a paper that requires Policy A must also be willing to follow Policy A."

A hammer can be used to build a house, or to kill a person. We have a lot of history, law, and culture (likely more), around using tools like hammers so that we know what is good use vs what is bad. The above applies for many others tools as well.

LLMs can be very useful tools. However we also know there are a lot of bad uses and we are still trying to figure out where there are problems and where there are none.

They agreed to the no LLM policy.

> what's the problem?

Read the article. They self-selected into the no-LLM group and then copy/pasted from an LLM. Not only dishonest but just not smart.

Reading the article is exhausting. If I can leave a comment just as well without reading the article, then what's the problem? If I got something wrong, other people will point it out. That's a more efficient use of my time.

/s

Not to water down the snark, but isnt cause of situation described in the article the exact mentality you are mocking?

I believe that's the joke, yes.

This has nothing to do with whether it is ok to use AI or not, it is about whether it is ok to lie about using it.

The issue is not the tool use - research is a small community and violating submission terms is gonna get you stuck in the naughty corner.

Depends on the field. One of the most influential papers in economics was found to be incorrectly constructed with signs pointing to just straight up fraud. Basically it didn't include data that it said it did, which when included reverses the conclusion. Then when the authors were called out, they doubled down offering up the explanation that the conclusion again reverses if you add a third set of cherry picked data, followed by dragging the person calling them out through the mud in a NY Times opinion piece.

Those authors are still extremely prestigious professors in the field, and have suffered essentially no penalty. https://en.wikipedia.org/wiki/Growth_in_a_Time_of_Debt

All due respect this is by no means one of the most influential papers in economics.

It literally crushed economies and guided international monetary policy for at least a decade.

There's a reason why it's one of the only economic papers that has its own wiki page.

Not influential on economics research, but on economic policy.

Banned from doing free work?