News for hackers. Stuff that matters.
Re:A new replication crisis: Research that is less likely to... by seibelj on Friday May 21, 2021 @10:03PM (#27242576)
What’s crazy to me is nothing should stop an intelligent person from submitting papers, doing research, etc. even outside the confines of academia and having a PhD. But in practice you will never get anywhere without such things because of the politics involved and the incestuous relationship between the journals and their monetarily-uncompensated yet prestige-hungry army of researchers enthralled to the existing system.
Re:A new replication crisis: Research that is less likely to... by lnwlebjel on Friday May 21, 2021 @10:22PM (#27242715)
If you add 'self funded' to this hypothetical person, then it would not matter if they play any games. Getting published is really not that hard if your work is good. And if it is good it will get noticed (hopefully during the hypothetical person's lifetime). Conferences have less of these games in my experience and would help.

Also, I know of no researchers personally who are enthralled by the existing system.

Re:A new replication crisis: Research that is less likely to... by seibelj on Friday May 21, 2021 @11:23PM (#27243107)
Can you name a single person with a high school or BS degree published in nature or other high impact journals? If not, why is this the case?
Re:A new replication crisis: Research that is less likely to... by dr_kiszonka on Saturday May 22, 2021 @12:32AM (#27243500)
I think one of the most famous examples is that of Grosset, who published his work on statistical significance under the pen name "Student.”[0] I wish I could give you a more recent example, but I don't pay attention to authors' degrees much, unless a paper is suspicious and from a journal I am unfamiliar with.

If I am reading between the lines correctly, you are implying there are few undergrads publishing in high caliber journals because of gatekeeping. As a reviewer, I often don't even know the authors' names, let alone their degrees and affiliations. It is theoretically possible that editors would desk reject undergrads' papers, but: a) I personally don't think a PhD is required to do quality research, especially in CS, and I know I am not the only person thinking that; b) In some fields like psychology and, perhaps, physics many junior PhD students only have BS degrees, which doesn't stop them from publishing.

I think that single-authored research papers by people without a PhD are relatively uncommon because getting a PhD is a very popular way of leveling up to the required expertise threshold and getting research funding without one is very difficult. I don't suspect folks without a PhD are systematically discriminated against by editors and reviewers, but, of course, I can't guarantee that this universally true across all research communities.

0. https://en.wikipedia.org/wiki/William_Sealy_Gosset

Re:A new replication crisis: Research that is less likely to... by seibelj on Saturday May 22, 2021 @01:25AM (#27243761)
I believe that “good” research, i.e. that which would be referenced by other “good” researchers and useful in obtaining government grants, reported in the press, and so on is indeed gatekeeped. Some subjects such as mathematics and computer science have had much progress in preprints and anyone can publish anonymously and make a mark. But the majority of subjects are blocked to those already connected, especially soft sciences like sociology, psychology, and economics.

I think the entire academic enterprise needs to be burnt down and rebuilt. It’s rotten to the core and the people who are providing the most value - the scholars - are simultaneously underpaid and beholden to a deranged publishing process that is a rat race that accomplishes little and hurts society. Not just in our checkbook but also in the wasted talent.

Re:A new replication crisis: Research that is less likely to... by seoaeu on Saturday May 22, 2021 @03:46PM (#27249706)
The status quo isn't perfect, but I think you are severely exaggerating how bad things are. The fact that nearly all scientific publishing is done by people who are paid to do research (grad students, research scientists, professors, etc.) isn't evidence of gatekeeping. It just means that most people aren't able/willing to work for free.

It also isn't any sort of conspiracy that government grants are given out to people with a proven history of doing good research, as evaluated by their peers.

Re:A new replication crisis: Research that is less likely to... by Der_Einzige on Saturday May 22, 2021 @12:24PM (#27248006)
I personally as a BS holder only along with a (at the time) high school senior published a paper in a top 6 NLP conference. I had no help or assistance from any pH.d or institution.

Maybe not quite as prestigious as nature, but NLP is pretty huge and the conference I got into has average h index of I think 60+

Proof: https://www.aclweb.org/anthology/2020.argmining-1.1/

Re:A new replication crisis: Research that is less likely to... by dmingod666 on Saturday May 22, 2021 @12:33AM (#27243507)
In the same vein though, can you think of any person that wants to publish research and is actively being denied being able to publish..

The people you mention are probably making YouTube videos and writing blog posts about their findings and are reaching a broader audience..

Re:A new replication crisis: Research that is less likely to... by TimPC on Saturday May 22, 2021 @07:38AM (#27245546)
I know a person who got published in high school. They did so by working closely with multiple professors on various projects. You don't have to do a PhD to do this especially if you're a talented and motivated youngster.
Re:A new replication crisis: Research that is less likely to... by vnorilo on Saturday May 22, 2021 @03:00AM (#27244221)
I have personally recommended for publication papers written by people who do not have a master's degree. In most cases I did not know that at the time of review, but it did not occur to me to care about it when I did.
Re:A new replication crisis: Research that is less likely to... by dmingod666 on Saturday May 22, 2021 @12:39AM (#27243544)
Myers Briggs test is an example, pseudo scientific tests with questionable origin..

"Neither Myers nor Briggs was formally educated in the discipline of psychology, and both were self-taught in the field of psychometric testing."

Re:A new replication crisis: Research that is less likely to... by angrais on Saturday May 22, 2021 @08:39AM (#27245932)
I can name several high school students who conducted studies and led first author papers to leading HCI venues. They were supervised by academics though. Would that suffice?
Re:A new replication crisis: Research that is less likely to... by cycomanic on Saturday May 22, 2021 @03:05AM (#27244240)
This has nothing to do with gatekeeping. I agree that the current publication and incentive system is broken, but it's completely unrelated to the question if outsiders are being published. The reason why you see very little work from outsiders is because research is difficult. It typically requires years of full time dedicated work, you can't just do it on the side. Moreover, you need to first study and understand the field to identify the gaps. If you try to identify gaps on your own, you are highly likely to go off into a direction which is completely irrelevant.

BTW I can tell you the the vast majority of researchers are not "enthralled" by the system, but highly critical. They simply don't have a choice but to work with it.

Re:A new replication crisis: Research that is less likely to... by dash2 on Saturday May 22, 2021 @01:46AM (#27243876)
I think this is a bit naive. One thing that stops a smart person doing research without a PhD is that it takes a long time to learn enough to be at the scientific frontier where new research can be done. About a PhD’s length of time, in fact. So, many people without a PhD who try to do scientific research are cranks. I don’t say all.
Re:A new replication crisis: Research that is less likely to... by TimPC on Saturday May 22, 2021 @07:36AM (#27245538)
Some quality journals and conferences have double blind reviews now. So the work is reviewed without knowing who the work belongs to. It's not so much the politics of the system as the skills required to write a research paper being hard to learn outside of a PhD. You need to understand how to identify a line of work in a very narrow field so that you can cite prior work and demonstrate a proper understanding of how your work compares and contrasts to other closely related work. That's an important part of demonstrating your work is novel and it's hard to do (especially for the first time) without expert guidance. Most students trying this for the first time cite far too broadly (getting work that's somewhat related but not related to the core of their ideas) and miss important closely related work.