LOL! No. Ratings are nearly useless. By which I mean I don't know how I would incorporate ratings into my own research.
How do you keep a ratings system from being a proxy for popularity, rather than a measure of novelty or utility? How do you keep it from being easily gamed? Does a "5.2" for a 1995 paper have the same meaning as a "5.2" for a 2010 paper?
"You want peer recognition so why not go to your peers directly instead of a proxy like these publications"
Do tell me how. Post on my blog and hope people come across it?
Journals outsource marketing. Get rid of journals and something must replace that service, or I have to become a marketer.
"This system is a product of a bygone era .. relying on copyright"
You are mistaken. The system does not rely on copyright. The scientific publication system dates from the 1600s, dated from Oldenburg's 'Philosophical Transactions of the Royal Society'. Copyright dates from the 1700s, dated from the Statute of Anne.
The scientific publication system depends on reliable selection and curation. Its current form is dominated by large publishing companies which use copyright law for revenue generation, but that's only been true for the last few decades. Scientific publication also existed in the USSR even without copyright protection. (article 103(2) according to https://en.wikipedia.org/wiki/Copyright_law_of_the_Soviet_Un... gave "permission to reproduce published scientific, artistic, or literary works as excerpts (or even entirely) in scientific, critical, or educational publications").
"most of these papers were made with public funding so there shouldn’t be any copyright on it to begin with the copyright belongs to the public domain they payed for it."
I understand and sympathize with your argument. But in practice things aren't so clear cut. Rarely does the money come 100% from public sources. Is your argument that even 1 cent of public funding means the entire work product must be in the public domain?
Are you making the moral argument that copyright should to the funding source? Or is it limited only to public funding sources? For example, if 100% of the money for a project comes from the Bill & Melinda Gates Foundation, then do you want the foundation to control the copyright the work product? What if it's 80% Gates and 20% public funded? Is it public domain then?
If 3 years of research was publicly funded, and resulted in paper, and 2 more years was privately funded by BigCorp., to analyze the data further, then must that additional two years also be in the public domain?
And on and on. Unless "1 cent => public domain always", then there's no clear line. Is that your argument?
(BTW, there is another clear line - work by a US government employee is in the public domain. A small percentage of the scientific papers are in the public domain because of that.)
To respond to this particular point, I'm not familiar with any characteristics of the academic journal system which prevent these problems either. Name recognition is often key to getting into competitive journals, which is just another way of calling it an easily gamed popularity contest.
He was later admitted to Cambridge for a PhD and sent the exact same paper (same formatting, content, layout but different department/uni). It was immediately accepted. Does it necessarily mean that scientists in poorer countries are doing bad science? Or just that they didn't get their visa / couldn't leave their country / just want to stay there, etc.
The first filter most editors use is: "Do I know this research group? Are they part of the club? Have they published in top journals?". If not, it won't get read (unless it's a compelling and exciting title and abstract).
Double blind doesn't always work, sometimes it's painfully obvious who the PI is, but it should stop stuff like this from happening.
> We are no longer able to accede to requests from authors that we withhold their identities from the referees. Such "double-blind" reviewing has been discontinued. [0]
[0] https://journals.aps.org/prl/authors/editorial-policies-prac...
But the argument is that "giving other researchers the ability to rate ... [would work] just as well as this journal reputation system".
My question is, simply, why should I believe this?
Bibliometrics often look at citations. Citations are, yes, a marker of popularity and social connections, but also have certain scholastic requirements. Editors and reviewers can and do point out missing references or highlight too many irrelevant self-citations. This is a type of moderation system. Imperfect certainly, but it does provide some grounding.
Also, it's not like the scientific publication system only contains original research papers. Review papers fill a curational role, with the advantage of a single voice (even if from co-authors) making the comparisons, so I at least have a relatively constant baseline.
On the other hand, I've always been suspicious that people can reliably guess the authors when doing a blinded review. I've seen one study that said 30% of reviewers (in a small sample) could do so, but that hardly seems like a reason to abandon the idea altogether.
I don't see a problem with this. If you want public money, you have to put your work in the public domain. That's just a condition of getting the money.
The NIH does this. If you receive a single penny from an NIH grant, the work must be posted to PubMed.
So if you publish a paper titled "XYZ" in a journal and also put it on your blog, making sure your blog is visible to search engines - anyone searching for the paper's title "XYZ" will find it for free on your blog?
(I'm not involved with the science community)
Note that my comment concerned the proposal "why not go to your peers directly instead of a proxy like these publications". Your proposal replaces "instead" with "in addition to".
I do think open commenting systems would be a boon to academic discourse though.
> When Fourier submitted his paper in 1807, the committee (which included Lagrange, Laplace, Malus and Legendre, among others) concluded: ...the manner in which the author arrives at these equations is not exempt of difficulties and [...] his analysis to integrate them still leaves something to be desired on the score of generality and even rigour.
https://en.wikipedia.org/wiki/List_of_important_publications...
> What's wrong with giving other researchers the ability to rate and comment on a paper.
I do not see nothing inherently bad with this (provided very good moderation tools are in place). However...
* > I would argue it works just as well a this journal reputation system *
As a journal reputation system, possibly; although I find that experts have often very different taste compared to say general Hacker News public -- for instance, machine learning has a very high "coolness factor" these days compared to other areas of computer science, but it would be unwise to make a system where a weaker paper on machine learning gets vastly more credit than a better paper on say approximation algorithms.
However, an important part of the journal submission process is the fact that the editors ask the members of the academic community for spending several days on the paper, noting major and minor details and checking the validity of the proof.
Since the reviewers do their work for free, there is no reason why there cannot be an open system that replicates the same process; still, just "adding a comment system to arXiv" will probably not be enough, as people spend rarely that much time on comments (15 minutes tops, not entire days).
First, the price. The last paper I reviewed took me about 10 hours. I charge $200/hour. Should I charge $2000 for my time? If I'm paid by the hour, who tells me when to stop? If I'm paid a flat fee, how much is that fee?
I have seen a reviewer write something like "I read the paper twice. It seems good. Publish it." Does that get the same as someone who spends 10 hours for a detailed critique?
Second, does payment constitute work-for-hire? Does the publisher and/or reviewer need to pay taxes? Might some reviewers have a contract which prevents them from taking on part-time work? What if the reviewer is overseas? Do tax agencies from two different countries need to be involved?
Third, most people expect to publish, and get their papers reviewed. If a single author has three reviewers, then that author should expect to do about three reviews. This is an imperfect system which rewards publishing rather than reviewers, but there are also some compensations to the reviewer.
Moreover, I recall one story - I don't know the validity of it, but it has a ring of truth - where a daycare got annoyed with parents who picked up the children late. They decided to fine parents should that happen. As a result, children were picked up late more often because the fine was interpreted as a fee for extra daycare, and no longer social chastisement.
I suspect that reviewers are in a similar boat. I'll feel different about it if I get paid a pittance than if I were to do it for free, so long as others do the same for me in the future.
Of the people I meet at a conference, how do I figure out who has been a reviewer for Nature, and how do I verify that claim?
What's wrong with giving other researchers the ability to rate and comment on a paper.
It's a not a perfect filter but I would argue it works just as well a this journal reputation system and it prevents issues like people having to publish with them to get recognized not to mention it encourages discussion between academics which could lead to some breakthroughs as well.
You want peer recognition so why not go to your peers directly instead of a proxy like these publications.
This system is a product of a bygone era and as all other systems like it it is resisting change by relying on copyright.
I see copyright as the root of most of our current societies issues it is just not compatible with how we work not to mention most of these papers were made with public funding so there shouldn’t be any copyright on it to begin with the copyright belongs to the public domain they payed for it.
A publication just publishing it on their website and slapping a price on it is pure theft and they should be treated like the pirates they are pursuing.