And yet the peer review process relies on unpaid volunteers. Sure, there's some kudos. But the publishers are laughing all the way to the bank.

> They provide an objective, systematic evaluation of research

Publication bias is a thing. A hugely significant thing.

> Piracy is not an alternative to publishing, so it isn't a competitor

This is an odd semantic nitpick without any discernable meaning. I feel very certain that many who benefit from working within academic publication see piracy as a threat to their revenue streams, hence "competition".

> piracy destroys the market

No, the market has simply changed.

> We need to collaboratively imagine and build a better system for communicating and evaluating science

On this I completely agree.

Nobody is disputing that Elsevier et al make a profit that isn't entirely deserved. You don't have to argue that point.

But that glosses over the fact that their revenue is absolutely not pure profit. PLOS shows that there are costs associated with publishing, even if you get a lot of the work for free. Those $2000 are the value that publishers provide. Maybe they're charging $5000 for those $2000 right now, and that's morally suspect. But when that business model is destroyed, someone has to come up with a way to finance those $2000 of value by other means.

Also:

> Publication bias is a thing. A hugely significant thing.

"Publication bias" isn't the arbitrary injection of a publisher's opinions into articles or something like that. It just describes the fact that more surprising, or positive results are more likely to get published.

Doing away with publishers won't have any effect on this mechanism. Any sort of ranking or discovery mechanism will ultimately prefer the more spectacular result, because that's what people are looking for. In the specific case of pharmacology, mandatory pre-registration of trials, and all-encompassing meta-analysis of those is a good strategy, and it's completely orthogonal to the process of publishing the paper.

By your logic, I can "compete" with GM by stealing all of their car inventory and blowing up all their factories in Detroit. I don't think that makes sense.

> the market has simply changed

Strongly disagree. What exactly is the new market? Selling journal t-shirts?

> yet the peer review process relies on unpaid volunteers

> Publication bias is a thing

I say that publishers add much more value than pirates. Sure, publishers aren't perfect and others add value too. So what?

When the market can't compensate authorship per copy this does not mean the author cannot exist, but that authors will be compensated differently.

Do you seriously think that people would stop making music because they can't become millionaires? There's tons of people making a living playing on YouTube, filling the niche, and living off endorsments and side-selling of merch or lessons.

How do you think your analogy would apply to scientific publishing?

want it printed on dead trees? then sure, pay for a hard copy.

researchers work 40+ hour weeks for months or years. the majority of the work in publishing is reviewing that research, which is outsourced to peers for free. the actual publication is a tiny part (time-investment wise) of the total value.

Further, peer reviewers review for free because they can afford to--they are being paid by government and industry. Publishers cannot survive without being paid.

If academics can peer review, they can do the analysing of reviews and accepting/rejecting, too. Even for-profit publishers often put all that work on an unpaid editorial board.

Also, the costs do not have to be so high that they make Wiley or Springer-type for-profit journals the only way of publishing scholarship. In my own field, the most respected journals are printed by learned societies founded on the basis of an endowment or national academies of science. Since the labour and coordination is already done for free here by academics -- just like with many for-profit journals -- the only additional costs are typesetting and coordination with the printer/distributor, and it turns out that these costs are not so high that you have to lock your content down. These journals do not even have to charge submission fees like other journals in the "open access" movement claim is necessary; it just all doesn't cost that much.

So, piracy is a great solution in the short term, and in the long run there should be a return of journals to learned societies. After all, that is where many of them were originally published anyway before the for-profits bought them up (promising to save on labour and ensure better quality, but in fact unpaid labour is still used and the quality of editing and typesetting has dropped drastically now that for-profit publishers often demand camera-ready copy with no quality inventions of their own).

all this tells me is that there's a serious lack of adequate tools or some common, open platform to facilitate this. it was hard [expensive] to collaborate on sourcecode before Github; now it isnt. it was difficult to research prior art for patents, now it isnt via patents.stackexchange.

Except it needn't be. Reddit-like voting and commenting site for scientists would work just as well to vet papers and provide valuable feedback to authors with nearly zero overhead.

Probably not. There's a well-known essay that explains why karma/votes/likes work very well for easy tidbits like cat photos, gifs, and puns. (The stuff that takes less than 5 seconds to evaluate and requires near-zero effort to judge whether one "likes" it or not.) Unfortunately, I don't have a cite because I forgot who the author was.

Longer topics such as a 20000 word dense research paper with supporting data that requires at least 2 hours of study by a reader/reviewer doesn't work for a karma points system. The effort is too high to contribute significant # of votes. It's the same difficulty for other long form content such as manuscripts for novels/books.

Instead of karma voting, these works require curation that passes the "signal of approval" from one person to the next. With book manuscripts, curation is first done by the literary agents and then the acquisition editors. With scholarly papers, it's a somewhat analogous process (committee 1st pass then forwards to the peer reviewers). Each "approver" has a reputation which signals to the next person that it's ok to invest the hours into studying the work. This social signaling mechanism is also similar for startup entrepreneurs' business plans and VCs. VCs would rather be exposed to your business idea from a referral rather than an unsolicited cold email. All 3 domain examples of book manuscripts, academic research, and business plans share the same theme: people have limited time to wade through a vast sea of poor quality content.

I think people can conclude that flipping through 100 cat photos is different than slogging through 100 dense papers. Another example of this dynamic is the Stackoverflow voting system. There have been several meta discussions lamenting the fact that the "easy" questions attracts a high volume of activity (answers + votes) but the "hard" questions languishes with little attention. That's because the easy questions are more analogous to the low-effort cat photos.

Another commenter in the thread mentioned that github solved workflow for programmers and the same could be done for peer reviewing academic papers. This is misunderstanding the underlying issue.

Evaluating dense papers is not an "information technology workflow problem" that's waiting for collaboration software to solve. The bottleneck is human effort to properly read papers to judge its correctness and significance. In 1687, a human would read Newton's Principia Mathematica at ~200 words-per-minute and after several hours, maybe he'd conclude that "yes, F=ma looks reasonable here". It's 300 years later and we're still the same slow humans reading papers at ~200 wpm. It's worse now because there's a larger glut of bad papers to wade through than before.

Peer review of novels, scientific papers, business plans, is not solvable by a Github/Sharepoint/DropBox. The bottleneck isn't the copying of pdf or LaTeX files from one person to the next. It's the hours of human effort to judge the papers and signal to the next person to also expend hours of human effort.

The issue about publishers isn't the filtering. The real issue is that they they charge too much money for filtering. The high profits is misleading some people to think it's an easy problem solved by github or a reddit-like voting system.

You're attacking a straw man. I said reddit-like, not reddit-clone. There are plenty of obvious problems with reddit's voting system. So what?

What is scientific consensus if not a vote among some random sampling of scientists studying a field. What's left is to formalize this process in an open system so we're not beholden to the publishers and other parties with perverse incentives.

> Evaluating dense papers is not an "information technology workflow problem" that's waiting for collaboration software to solve. The bottleneck is human effort to properly read papers to judge its correctness and significance.

You're wrong. Firstly, it's often impossible to judge the significance of a result even soon after publishing.

Secondly, reviewers already donate their time to review papers, so the human effort is already there. What's needed is a well-recognized system encoding a publishing process, whereby some random selection of scientists obtain access to early drafts with a way to provide commentary and ratings on various metrics, with final publication after some quality consensus is reached among the random selection of reviewers (as a rough idea).

This is exactly an information technology problem.

Yes, and that's what I understood you to mean and a reddit-like voting system for papers is what I was talking about. (I even wrote "reddit-like voting system" as my last sentence in the post you replied to.)

>You're wrong. Firstly, it's often impossible to judge the significance of a result even soon after publishing.

I'm not talking about "significance" in relation to the whole of human history. I mean "significance" in the sense of a threshold for requesting other people to invest their time & effort to study the paper and also the subsequent subscribers to read.

>, reviewers already donate their time to review papers, so the human effort is already there.

Yes, but the key you're missing is that they donate their time because a respectable _somebody_ requested them to do so. The triggers for motivating that donated effort matters.

If I'm a working scientist working 50 hours a week, I'm not logging into a reddit-like voting system and investing hours to read (mostly bad) papers that have 0 votes so I can submit my 1 vote. No, somebody I respect (e.g. a journal editor) needs to ping me and say, "hey, this paper looks interesting, can you review it?" After I peer review it, I may get some quid-pro-quo such as free access to the journal's other papers and/or when it's my turn to submit a paper, the editor will consider forwarding it. So yes, I'm not "paid in cash" but I'm paid "in kind" with some intangibles.

That social dynamic is not replicable in a reddit-like voting system.

Therefore your following scenario of:

>whereby some random selection of scientists obtain access to early drafts

... is not realistic because scientists do not want to be part of random pool to be assigned random papers. That's contrary to basic human nature! Scientists want papers referred to them by another human and not an algorithm. It's not a technology problem, it's a human problem.

All the software suggestions to solve this from programmers including ideas such as a reddit-like system, or a github-like system, or a blockchain type system, etc ... they all misunderstand the culture and psychological motivations of publishing.

>What is scientific consensus if not a vote among some random sampling of scientists studying a field.

Yes, I can see that if you reduce the problem to the simple abstraction of "votes", it seems like building a software platform to formally capture votes is The Solution.

What your simplification leaves out is all the human signaling required before people render their votes. That's the human dynamic that collaboration software doesn't solve.

(In any case, an accepted published paper isn't "consensus" per se; it's just another piece of evidence for people to weigh. The paper just passed through enough human curation filters that may be worthwhile to read.)

I agree with you that it is difficult to improve on the peer review process. At the same time, several researchers have pointed out that publishers aren't really involved in the peer review process. It should be possible to coordinate journal publication through a software platform, replacing the publishers without affecting the peer reviewers or editors.

Yes, I agree employees at Elsevier aren't doing the actual peer review. However, they do critical filtering work before the any journal editors see the paper. See example from Elsevier employee Angela Kerr.[1]

The programmers who think they can revolutionize academic publishing by developing a new peer-review collaboration platform don't even realize that people like Angela Kerr exist. (See repeated disgust in this thread about "peer reviewers are unpaid". That's true but it also makes Ms Kerr's paid work invisible.)

In addition to basic copy-editing, she prevents crackpot articles such as "Darwin's Theory of Evolution is proven wrong by Trump" from reaching the journal editors and wasting their time. (She may inadvertently forward some bad articles but she has enough education to reject many of them outright to minimize the effort by the journal's scientists.)

Administrative people like her cost money. One would have to replicate her duties somehow in the new software platform. That's because the journal editors (scientists) are busy and they absolutely will not want to log into a new collaboration platform and stare at a long queue of unfiltered papers that everybody uploaded. Even though Angela doesn't do the "peer review" per se, the journals still want her to make a first pass cut. None of the ideas I've seen from programmers integrate her function into their model of what peer-review software should do.

With something like Stackoverflow, you have volunteers scrolling through the queue of questions and deleting/editing bad ones so their highly valued experts don't have to waste time with them. That's workable for "programming questions" because they are typically less than 100 words. Even with smaller workload of not reading 20000-word research papers, volunteers still get burned out and quit.

For highly prestigious papers like Nature and Cell that would attract a high volume of uploaded papers, it's very doubtful you'd have volunteers like Stackoverflow to slog through all the junk. Therefore, you'd have to staff up some copy editors and pay them money to do unpleasant work.

The problem is that Elsevier charges an insane amount of money for subscriptions (~$800 million in profits with 36% profit margin). Yes, a new system can charge much less money and be more efficient but you still have to do the work that Elsevier does. The journal editors don't want to do it. The potential software model of synthesizing ideas from reddit+github+dropbox+bittorrent+Sharepoint+blockchain ... don't account for Angela Kerr. Programmers think only in terms of distributing files like pdf and LaTeX. That's why they think academic publishing is a solved software problem and not a human effort problem.

>It should be possible to coordinate journal publication through a software platform, replacing the publishers without affecting the peer reviewers or editors.

The software platform can replace Elsevier in particular, but it can't replace the fundamental "work activity of a publisher". Angela Kerr (or whoever employs her) wants to be paid to do the unpleasant work that journal scientists don't want to do. And she doesn't want to be paid in reddit-like "karma votes" or github-like "stars". The new system can pay Ms Kerr less money than Elsevier (and hopefully pass those savings on to subscribing libraries) but let's be clear that the journal scientists absolutely do not want a new software platform without her.

[1] https://www.elsevier.com/connect/confessions-of-a-managing-e...

There is great need for a software nonprofit to enable scientists and editors to collaborate with minimal overhead, because even $2000/paper is too painful. The true marginal cost should be under $200/paper for scientific editing,[1] and at that price point, open access will overwhelmingly win. If we can build such a platform, I believe that Elsevier will find it very difficult to retain their journal editors.

[1] http://www.biomedicaleditor.com/pricing.html