Probably the best proxy for quality is popularity that endures for centuries since we can’t measure quality directly. Either way, if there’s a max of one paper per year, scientists will be trying for the most impactful work.

People are already boosting their impact scores by requiring papers they review to add citations for their past work.

Performance Metrics always result in people gaming the system, and impact scores have quickly become unreliable.

At the same time, reviewers are chosen based on their expertise in a field, and one of the things reviewers look for is that new papers cite relevant prior research. Are you suggesting that reviewers should just let papers go through without citing relevant prior research if it is their own research? Does it actually matter whose research it is as long as it is relevant prior research?

Of course it matters, adding citations to already written research is deceptive and thus actively harmful.

The point of citations isn’t simply an index of prior research it’s there to inform readers of the writers context.

When a paper is rejected because it doesn't contain references to relevant prior research, the citations shouldn't simply be added; the relevant prior research that wasn't considered should thus be considered, and the paper should be reworked with that new context in mind.

Research should be put into the proper context in the field, not based on what the writer personally looked at. If a researcher's context is myopic enough not to include relevant prior work known or done by reviewers, then the research itself is of questionable value. If someone submits work that purports to be novel, yet they haven't done a proper literature review to include relevant studies, I'd say that is itself deceptive and more actively harmful to the field as a whole.

Suppose an experiment takes 5 years, nothing written after the experiment begin should have any barring on what was done and the resulting data.

The same is true to a lesser extent for any research. Now if the field has discovered some flaw in what was done then things go well beyond the simple need for a citation.

That's not how competent research is done. You don't go into a hole for 5 years and then come back into the world with a fully-formed paper. While you're doing that research, the world moves on, and if your citations are from 5 years ago, any publication will ask the extremely appropriate question "How does this matter w.r.t. the research published since you started this work?", and send the paper back for rewrite. Or, it may ask the even more appropriate question "Why does your research matter given the results of papers x,y,z which are directly related to your research question?" Any competent researcher should preempt in their literature review, and if they can't then it's not research but instead PR.

And this doesn't even touch upon the fact that research that's going to take 5 years is going to be well-funded, and you'll have to write interim reports over that period. When you get 5 years of funding, you don't get that all at once. During that time, you'll have to answer for advances in the field and justify why the research is still worth doing.

Experiments that take a long time aren’t always expensive, sometimes you just need time. What actually happens to a sample held at X condition for some period of time?

As to other field moving on, it can be a reason to reject the paper in it’s entirety but has zero impact on the results which is the only thing that actually matters. Papers aren’t textbooks, the goal is to communicate what was done and what happened not inform the reader of some wider context.

In the end if I am reading 30 papers on some topic I really want them to be as short and clear as possible. And for topics I keep up with I sure don’t need this fluff.

> Papers aren’t textbooks, the goal is to communicate what was done and what happened not inform the reader of some wider context.

And a literature review is part of what was done, always. If what you said were true, there would be no point in a literature review at all. Instead, every paper includes one because researchers understand it's an intrinsic part of research. That's what makes it research instead of just search: research is a systematic process that involves, in every definition of the process I've ever come across, a literature review. I challenge you to find an example of peer reviewed research that doesn't contain one.

Here’s a translation of a rather fun counterexample.

https://cdn2.hubspot.net/hubfs/232514/Einstein%20E=mc2%20(pp...

But, I don’t disagree that someone should keep up with the literature. Including it as part of a paper is what I disagree with.

It’s unlikely this paper was peer reviewed. The peer review process was much different in Einstein’s time. It wouldn’t pass for research today. I kind of thought we were talking about how research is conducted in modern times, not how they did it over 100 years ago.

If you want a more recent example. How about 1966 with the counter example of Euler’s Conjecture on Sums of Like Powers.

A direct search on the CDC 6600 yielded 27^5 + 84^5 + 110^5 + 133^5 = 144^5

as the smallest instance in which four fifth powers sum to a fifth power. This is a counterexample to a conjecture by Euler [1] that at least n nth powers are required to sum to an nth power, 1 > 2.

And that’s the paper. They could have gone into great detail about the program used etc, but it clearly communicates what was done and the results so why bother.

“I consulted a magic box and arrived at the answer” is precisely the kind of paper that is an example of search, and not research. I mean, they plainly stated that’s what they’ve done.

How would you go about redoing exactly what Lander and Parkin did? You can’t because they don’t really say. Isn’t this whole thread about a lack of reproducible studies resulting in fraud?

But anyway, I like what you’re doing here because you’re beginning to document why the modern peer review system was developed in the first place in the 70s. Sounds like a good topic for a research paper.

A counter example is a proof unto it’s self there is no need to derive it again, just verify the counter example is correct.

Many modern papers are of this form. If you have some specific data as recent enough to be valid I could provide one but I am not going to just pick a paper from 1990 and you say nope not recent enough.

There are many reasons to reproduce papers, it doesn’t matter if the author is unquestionably correct. One is to establish that they actually did what they said they did. What if the authors stole this result? It’s certainly happened before. If we don’t accept students divining solutions, why should we accept that from scholars?

Another is for posterity and for future generations. If I give the paper you cited to a student, they can be wowed at its brevity and impressed by its genius, but they learn nothing.

I get you’re trying to prove your point but all you’ve done is solidify my position in my mind that literature reviews are essential to good research. You’ve given me a great topic for my talk to our incoming class of Ph.D. students in a couple weeks, so thanks for that, I mean it. Cheers.

So you aren’t actually open to new information, that’s disappointing but unsurprising. I get your stance but again papers aren’t textbooks.

O well, there’s a reason science advances one retirement at a time.

Sheesh, I don’t think that was called for. Was a good discussion until that :/

Yea, I was going to delete it.

Just got annoyed you tried to weasel out of picking a date and tried to pretend you had won the argument.