As to why my lab might be preferring papers from big-name journal: I think 1) there's admittedly our own bias of wanting to cite big papers vs. small papers, 2) if we're doing some basic literature search, Google will also yield results that are already more popular. Really, at the moment, I struggle to see how our behavior changes about which papers we're citing with or without scihub.
One thing I would do a lot with Google Scholar is find a highly-cited, somewhat recent (past decade or so) paper and then click into its citations to find the very most recent related work. IIRC this interface was chronological so there was no big-journal bias and I found a lot of stuff in random journals. Low-friction quality-filtering through SciHub was very helpful for me in this process.
I worked in a theoretical field, so it's actually possible to assess the quality of a paper just by reading it. If you're doing something experimental you may have to lean on the big-journal filter a little bit more.
Under what conditions do you see that changing? You yourself say how useful it is, but discount the effect it has on the citation graph. Could we say the samething about arxiv?
This is exactly why research needs to be open and freely available. The esoteric, rarely cited papers are locked behind paywalls and away from search engines. And in doing so, their impact fades over time as the flock centers around a single vein of focus. The future should look like a net not a river.
I am not an academic and I read papers for their metalessons in areas that interest me. But without Sci-Hub, lots of research would not be accessible outside of its niche, esp if it was unlucky enough to not be in the right journal at the right time.
On hilarious pattern I see, is when a bigshot in a field publishes a new paper, there is a race to cite it by some nobody. Basically the academic publishing version of "first post".
> we found that articles downloaded from Sci-hub were cited 1.72 times more than papers not downloaded from Sci-hub and that the number of downloads from Sci-hub was a robust predictor of future citations
> The results suggest that limited access to publications may limit some scientific research from achieving its full impact.
My uncharitable read is they slapped an interesting yet unsupported conclusion onto a rather boring observation.
Edit: In particular, it seems to me that papers in their control group are available on SciHub (correct me if I’m wrong), just nobody bothered to download them in the time window they analyzed. Which directly contradicts the “limited access” part of their claim.
I was not sure about this, but if this is the case then their conclusion seems to be unrelated to their findings.
I just looked into the actual data set[1], and analyzed a few papers from it, e.g. [2] the first paper in scopus_nature.csv. They're definitely available on Sci-Hub at least at the moment [3], and there's no reason to believe they weren't available back then.
Also, here's what they say about the dataset, emphasis mine:
> As a quality control, we performed a random sampling of all the articles retrieved, excluding those already present in the first data set. As in this second data set, the number of Sci-hub downloads is precisely equal to zero, we regard it as a control group (nC = 4,015) from which we are going to estimate comparisons for our experimental group (nE = 4,646).
Downloads equal to zero != not available.
[2] https://www.scopus.com/inward/record.url?eid=2-s2.0-85016141...
Of course, that is hardly a surprising result. I think most authors have read most papers they cite (although things like reviewer/editor shenanigans could make this less than 100%).
Of course, such a pedestrian result does not grab attention.
I'm wondering about this too. What could be the possible other reason for the papers not being downloaded? Boring titles? Preprints easily available? Open access?
> and there's no reason to believe they weren't available back then.
Is it possible that the articles really were not available, as they used the data from sep-2015 to feb-2016, early version of SciHub?
That's within the realm of possibilities but sure doesn't sound like the case the way they described their methodology. That would have required a catalog of available papers on Sci-Hub too, but the dataset they used only contained download logs, so they couldn't have known whether a zero-download paper was available or not (unless I misunderstood what the first dataset is about).
This was still a garbage attempt to 'prove' this conclusion. And I don't have any faith the authors measured the true magnitude of these effects.
"Papers with more citations are downloaded more frequently on Sci-Hub" makes more sense to me than "Papers downloaded frequently from Sci-Hub get more citations".
The first data set contained all articles (nE = 4,646) that were downloaded from Sci-hub between September 2015 and February 2016 [24, 25]. The second data set was extracted from the Scopus database and contained all articles published in the selected journals within the same time period. From this data set, we excluded articles already present in the first data set.
Their methods lend to circular reasoning since the conclusion could also be "papers of high interest (cited more often) are more likely to be downloaded from scihub"
In this case, there's something that actually can help establish the direction of the causality arrow: time. If the citations precede the downloads, then sure, it's more well known papers get downloaded more. This papers show that the downloads precede citations though:
"number of downloads from Sci-hub was a robust predictor of future citations"
While it's not a watertight argument that downloads lead to citations, the theory is at least plausible.
(Yes I read the paper and I’ve seen all the other factors they considered, but those factors simply aren’t as good predictors/indicators of interest as an organic download number from any channel, whether it’s ScienceDirect or SciHub.)
Edit: To be clear, papers in their supposedly "limited access" (if we want to draw the same conclusion) control group appear to be available on Sci-Hub, i.e. as open access as their Sci-Hub group. Details in this comment: https://news.ycombinator.com/item?id=23710992