1. the parent raised questions in a neutral way. these questions seem essential for validating experimental design. why would peer reviewers present such questions in passive-aggressive ways, and how can we fix this?
2. could you kindly recommend services/consultancies to validate experimental designs? if not, would you be open to consulting and doing what you did here -- suggesting ways to control for key variables? experiments relate to cancer research. contact info in bio.
They are called CROs (contract research organizations). Pay them money and they will work on your experiments for you.
Peer review is obviously a complex and controversial issue, but some key points (at least in life/medical sciences) include:
A. Your reviewers are very often your competition. Reviewers are supposed to be subject matter experts in your area of research, and academic science is a small world. The other subject matter experts are exactly the people competing with you for grants, to finish projects first, for trainees, prizes, etc. (You can typically ask that specific people do/don't review your paper, but it's at the editor's discretion. Some fields are simply too small to take such preferences into account.) You can often identify your (supposedly anonymous) peer reviewers because they respond with a critique that your paper should cite some specific papers, and they are the common name on the bylines of those papers.
B. Peer review is uncompensated work by academics, very often done for for-profit publishers. Hard to be thrilled with that paradigm (though some scientists feel it's a reasonable 'academic duty').
C. The mindset in peer review is often more about gatekeeping the journal hierarchy than about simply ensuring good experimental design. Publishing in top journals is often a career-making achievement. It is incredibly common for reviewers to ask for (often very time consuming) additional experiments based on their opinions about what is interesting, what a paper in the journal should look like, etc. There is a bias where reviewers don't want someone else to have an easier time publishing in X journal than they did.
D. Not exactly a critique of peer review, but I think it's important to realize that peer review is not even intended to address one of the major problems in science – irreproducible results and/or scientific fraud. Reviewers have to take all data presented on face value. At best, peer review is simply a check against poor experimental design, errors in reasoning, and authors making claims stronger than what the data supports.
> 2. could you kindly recommend services/consultancies to validate experimental designs? if not, would you be open to consulting and doing what you did here -- suggesting ways to control for key variables? experiments relate to cancer research. contact info in bio.
As mentioned, CROs are the companies in this space, though I'm not familiar with any that focus specifically on vetting experimental design.
it appears that peer review may be ideal for limiting shoddy science, but at the expense of limiting breakthrough science.
can you imagine if tech operated with these dynamics?
apple, google, stripe, and many others would have never passed peer review.
apple: idea for mobile phone. peer review: please add a physical keyboard.
google: idea for search engine. peer review: the 28th search engine? please do something novel like yahoo.
stripe: idea for payments. peer review: AJAX is for kids. please be serious.
No, it's far from ideal for that. Just reread the grand-parent comment.
Btw, putting your paper on arxiv is an alternative to traditional journals.
They call them 'pre-prints' there, but it's essentially a model where you publish first and then have the peer review from any interested party.
In science sometimes the goal is instrumental value, but more often it is inferential insight where there isn't a simple 'it works or it doesn't' truth value and the role of methodology and review to control for sources of false positives and false negatives, misconduct, and unwarranted interpretation of data are important.
I'd argue that peer review aids breakthrough science overall, because where shoddy but splashy research slips through review, sometimes years of research effort and funding get funneled into avenues opened by such putative breakthroughs that turn out to have been bullshit all along. The misdirection into dead ends has the opportunity cost of the potential of making real breakthroughs.
I'd assume that's the case, otherwise I feel like there would be more Halderia and/or less viruses
Halderia would be 'lower eukaryotes'; we are 'higher eukaryotes'.
The idea of 'helping the halderia out' by increasing mutation rate is kind of funny; that would almost certainly kill them (this is largely how radiation kills us). You would need to be able to reliably and specifically introduce favorable mutations (i.e. those that increase fitness), which is far beyond current capabilities.
We’ve been taking advantage of the latter with selective breeding for millennia and more recently through inducing random mutations with techniques such as irradiation the green revolution was quite famously kick started by new cultivars that where developed by irradiating seeds and selecting those which led to plants with favorable characteristics.
When you need the population size to recover, you dial down the radiation a bit.
Ruby red grapefruit is an example you can probably find in your grocery store. I think they made it by irradiating cuttings for grafting.
As a human I accidentally ingest a lot of 'bad bacteria', it's just dwarfed by the nutritious (or delicious..!) stuff. I think that's what GP's getting at - distinguishing 'accidental'/sife-effect consumption from intentional; not necessarily that it's great sentient or evolutionarily designed intent.
Viruses form a part of a control system for themselves and for higher and larger life forms. If any system becomes unstable, or if one organism starts dominating or growing out of control, the other organisms have an evolutionarily created self interest in bringing things back into balance. Viruses play a huge part in this self regulation mechanism.
Eliminating viruses would have the unfortunate effect of decreasing overall system stability for biological systems at every level.
If you don't mind me asking, how do you deal with this kind of culture, where a normal process is being purposefully obstructed with such behavior from the reviewer?
Do you ever call people out who sprinkle their "reviews" with such passive aggression? What is their defense? What highlights does your profession have that make it worth putting up with this?
So...
> If you don't mind me asking, how do you deal with this kind of culture, where a normal process is being purposefully obstructed with such behavior from the reviewer?
You just come to expect it. I realize this isn't the best answer or even a reasonable one, but it's how it is and there's really no controlling it.
> Do you ever call people out who sprinkle their "reviews" with such passive aggression?
No. Your field is dominated by experts who have a clique and run a crony network of influencers; getting mad at one of them is a great way to ensure you never hit tenure track / get punished on papers in the future (many journals are not blinded, or even if they are, it's fairly obvious who wrote a blinded paper given the subject matter in a niche field and/or timing attacks on the paper's submission + researcher's social media posts on the topics).
Sometimes - most of the time in my field - you have no idea who the reviewers were. I think this is typical for most science fields.
> What is their defense?
They suffered by much worse hands; really, they're being nice. (That's what they tell themselves.)
> What highlights does your profession have that make it worth putting up with this?
Very few. I work for a for-profit company, so the research we publish helps bolster the company's image, can be used in marketing, and so forth. Going from zero to one feels amazing. One day you're a guy with a dream that you'll publish an influential paper someday and give back to science, the next you have that publication credit - maybe even lead author credit - and going from 1 to N is just nowhere as interesting as going 0 to 1... like most things in life.
For some of us, science is in our blood, and it's our calling. Whether we like it or not. Most of the time, we don't. But we do it anyway.
https://experimentalhistory.substack.com/p/the-rise-and-fall...
I won't butcher the piece with a poor summary but it's a critical look at the peer review process.
Unfortunately, inertia is a hell of a thing. Science is stuck with peer review for a few more decades at the very least. Many postgrads would do unspeakable things and commit various crimes to become a first author on a paper accepted into Nature, for example. It just means everything to academics.
arXiv is the biggest undermining threat to the professional peer review process, but in a weird way, it also bolsters it.
Anyway, at least Computer Science doesn't care much about academic journals. Hacker culture remains fairly strong.
How do you mean?
https://en.wikipedia.org/wiki/ArXiv#Moderation_process_and_e...
That just smells of bad science, and make it not surprising the mainstream is "lagging behind". People actively stifling real scientific advancement, instead of letting it flourish.
I feel bad for people that do have that purity of purpose that either have to put up with that kind of BS, and people that have been victims of said behaviour.
I appreciate very much people who have it in their blood, their passion, like i have music and singing in my blood. You know you'll get someone who'll do the best job possible, and care about what sort of science is being done. Thank you for being you
A huge part of the explanation is survivor bias IMO. The vast majority of undergrads who start in life science labs wind up leaving after a few months due to some bad experience (or at least lack of a sufficiently positive experience; life science is mostly failure). A large proportion of PhD students leave for industry jobs (tech, biotech, smaller subset to finance/consulting) because academic faculty jobs are very hard to come by and require making very little money for a long time. The only people left for the long haul are sufficiently motivated by the upsides (see below) to deal with the bureaucracy and problems of academic life.
> Do you ever call people out who sprinkle their "reviews" with such passive aggression? What is their defense?
Reviews are anonymous (this is now becoming controversial), and most people wouldn't jeopardize the acceptance of their paper just to call out a reviewer being an asshole. Slight saving grace is that the journal editor (who sits between authors and reviewers) has the final say, and can override unreasonable reviewers (in principle).
> What highlights does your profession have that make it worth putting up with this?
In fairness, some upsides if you make it: intellectual freedom (carte blanche to study anything you can get funded), sense of significance of advancing the frontier of medicine, freedom to work on and profit from startups if your tech is translatable (there is a surprising number of millionaire++ biology professors), very good job security, etc. Some would argue that many/most biotech companies take from 1 to N technologies that academic labs brought from zero to one. Some examples: mRNA vaccines, numerous cancer immunotherapies, CRISPR/Cas genome editing, recombinant insulin. Just look at the Nobel Prize in Medicine list – nearly all academic work.
Peer reviews are text, it is hard to accurately infer emotions or intention. What one person sees as passive aggressive another person might see as polite and deferential or even helpful.
Consider further that scientists come from all over the world and often have deep cultural differences, it becomes even less likely that you know the reviewer's intention.
Science is really hard and it's easy to fool yourself. Smart people want criticism to help validate and improve their work.
[1] https://www.jacionline.org/article/S0091-6749(22)01423-3/ful...
(1) could be addressed in experiments where the algae-consumers in question are studied in monoculture (purified and grown in test tubes) with virus added artificially. This experimental design excludes the possibility that any middle man is present; if uptake is still observed, it must be direct.
(2) Measure viral replication, perhaps in a similar monoculture experiment as above. If the viruses are infecting (exploiting) the Halteria, they will have non-negligible replication (they are stealing the Halteria's resources to advance their own replication). Viral replication could be assayed by qPCR (counting viral genomes) or similar assays.
(3) This would require biochemical studies to determine the mechanism of uptake of viral particles. Typically, viruses are taken up (endocytosed) by cells as a result of interaction with receptors on the surface of target cells (prominently CCR5 in HIV, Ace2 for the COVID-19 causing virus). Of course, what biological systems are `trying` to do begs that we anthromorphize Halteria (or at least evolution), but one could conduct an evolutionary analysis to see if Halteria have progressively evolved receptors that improve viral uptake efficiency.
As an aside, the discovery of virovores hints excitingly (albeit remotely) at the possibility of creating virus sink cells/technologies that could eliminate viral particles in humans. An important question will be whether viruses, which evolve far faster than eukaryotes like Halderia (let alone than humans), can turn the tides in the evolutionary arms race and become the exploiters of the Halderia. Perhaps it's reassuring that we exist (mostly) symbiotically with our microbiota, despite their far faster evolution.