News for hackers. Stuff that matters.
Re:Sci Hub repository torrents of scientific papers by nneonneo on Sunday May 20, 2018 @08:20PM (#17115444)
The total collection is 54.54 TiB, with 690 torrents as of writing. For preservation purposes, I put magnet links to all the torrents here: https://pastebin.com/zTAqS7wz

So, even if the torrents go away, the magnet links should still be usable.

(Edited: fixed link)

Re:Sci Hub repository torrents of scientific papers by userbinator on Sunday May 20, 2018 @09:34PM (#17115672)
What I find more amazing than the fact that some people have been so neighbourly as to share such a volume of information, is that this is still just a tiny fraction of all human knowledge. Scientific journals are certainly an important collection, but still miniscule in comparison to all the other books out there. I have collected ~20GB of automotive service manuals and related data, and that's a tiny amount too. To speak nothing of the many terabytes of entertainment others have...
Re:Sci Hub repository torrents of scientific papers by m-p-3 on Monday May 21, 2018 @12:12AM (#17116086)
I've seen some Plex servers hosting in the amount of petabytes of content, it's crazy.
Re:Sci Hub repository torrents of scientific papers by phyzome on Sunday May 20, 2018 @11:30PM (#17115964)
Downloading all the torrent files:

  mkdir torrents
  for i in {000..689}; do curl -sS "http://gen.lib.rus.ec/scimag/repository_torrent/sm_${i}00000-${i}99999.torrent" -o "torrents/sm_${i}00000-${i}99999.torrent" -m 30; done
...which themselves take up about 75 MB, so here is the full collection of torrent files for your convenience:

https://lab.brainonfire.net/drop/delete-after/20180630/torre... (please use the next link if possible, though!)

And a torrent that is basically equivalent to that file, thanks to the below commenter:

https://lab.brainonfire.net/drop/delete-after/20180630/scihu...

Each torrent appears to index 100 zip files (about 800 MB each), each of which presumably contains 1000... journal articles? I don't know. The seed isn't blazing fast, so it will be a bit before I can pull out a random zip file from this randomly selected torrent and inspect it.

Re:Sci Hub repository torrents of scientific papers by nneonneo on Sunday May 20, 2018 @11:50PM (#17116011)
I base64'd a torrent file containing the HTML index and all 690 current torrent files and stuck it here: https://pastebin.com/p1nN6veK. Would appreciate if you could upload that in a binary format somewhere.

Magnet link for simplicity:

  magnet:?xt=urn%3Abtih%3A97fc8218775e3a1e5b90607435c0581f839e0f1d&dn=repository_torrent&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.open-internet.nl%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.skyts.net%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.piratepublic.com%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2F9.rarbg.to%3A2710%2Fannounce&tr=udp%3A%2F%2Fpublic.popcorn-tracker.org%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker1.wasabii.com.tw%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.zer0day.to%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.xku.tv%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.vanitycore.co%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.mg64.net%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.tiny-vps.com%3A6969%2Fannounce&tr=udp%3A%2F%2Fp4p.arenabg.com%3A1337%2Fannounce&tr=udp%3A%2F%2Foscar.reyesleon.xyz%3A6969%2Fannounce&tr=udp%3A%2F%2Fopen.stealth.si%3A80%2Fannounce&tr=udp%3A%2F%2Fopen.facedatabg.net%3A6969%2Fannounce&tr=udp%3A%2F%2Fmgtracker.org%3A6969%2Fannounce&tr=udp%3A%2F%2Fipv4.tracker.harry.lu%3A80%2Fannounce&tr=udp%3A%2F%2Finferno.demonoid.pw%3A3418%2Fannounce
I am currently seeding this small file and will do so for as long as I am able.

(Edit: updated magnet link and torrent to include trackers)

Re:Sci Hub repository torrents of scientific papers by phyzome on Monday May 21, 2018 @12:23AM (#17116118)
Thanks! Uploaded as binary to https://lab.brainonfire.net/drop/delete-after/20180630/scihu... and currently seeding.
Re:Sci Hub repository torrents of scientific papers by Faaak on Monday May 21, 2018 @06:49AM (#17117416)
I can't manage to load this magnet link in transmission:

  Error adding "magnet:?xt=urn%3Abtih%3A97fc8218775e3a1e5b90607435c0581f839e0f1d&dn=repository_torrent&tr=udp%3A%2F%2Ftracker.coppersurfer.tk%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.open-internet.nl%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.skyts.net%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.piratepublic.com%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.opentrackr.org%3A1337%2Fannounce&tr=udp%3A%2F%2F9.rarbg.to%3A2710%2Fannounce&tr=udp%3A%2F%2Fpublic.popcorn-tracker.org%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker1.wasabii.com.tw%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.zer0day.to%3A1337%2Fannounce&tr=udp%3A%2F%2Ftracker.xku.tv%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.vanitycore.co%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.mg64.net%3A6969%2Fannounce&tr=udp%3A%2F%2Ftracker.tiny-vps.com%3A6969%2Fannounce&tr=udp%3A%2F%2Fp4p.arenabg.com%3A1337%2Fannounce&tr=udp%3A%2F%2Foscar.reyesleon.xyz%3A6969%2Fannounce&tr=udp%3A%2F%2Fopen.stealth.si%3A80%2Fannounce&tr=udp%3A%2F%2Fopen.facedatabg.net%3A6969%2Fannounce&tr=udp%3A%2F%2Fmgtracker.org%3A6969%2Fannounce&tr=udp%3A%2F%2Fipv4.tracker.harry.lu%3A80%2Fannounce&tr=udp%3A%2F%2Finferno.demonoid.pw%3A3418%2Fannounce": invalid or corrupt torrent file
Re:Sci Hub repository torrents of scientific papers by nneonneo on Monday May 21, 2018 @08:11AM (#17117779)
Sorry, just tried this and apparently Transmission isn't a fan of the encoded colons (despite that being very valid URL syntax).

Try this one instead:

  magnet:?xt=urn:btih:97fc8218775e3a1e5b90607435c0581f839e0f1d&dn=repository_torrent&tr=udp:%2F%2Ftracker.coppersurfer.tk:6969%2Fannounce&tr=udp:%2F%2Ftracker.open-internet.nl:6969%2Fannounce&tr=udp:%2F%2Ftracker.skyts.net:6969%2Fannounce&tr=udp:%2F%2Ftracker.piratepublic.com:1337%2Fannounce&tr=udp:%2F%2Ftracker.opentrackr.org:1337%2Fannounce&tr=udp:%2F%2F9.rarbg.to:2710%2Fannounce&tr=udp:%2F%2Fpublic.popcorn-tracker.org:6969%2Fannounce&tr=udp:%2F%2Ftracker1.wasabii.com.tw:6969%2Fannounce&tr=udp:%2F%2Ftracker.zer0day.to:1337%2Fannounce&tr=udp:%2F%2Ftracker.xku.tv:6969%2Fannounce&tr=udp:%2F%2Ftracker.vanitycore.co:6969%2Fannounce&tr=udp:%2F%2Ftracker.mg64.net:6969%2Fannounce&tr=udp:%2F%2Ftracker.tiny-vps.com:6969%2Fannounce&tr=udp:%2F%2Fp4p.arenabg.com:1337%2Fannounce&tr=udp:%2F%2Foscar.reyesleon.xyz:6969%2Fannounce&tr=udp:%2F%2Fopen.stealth.si:80%2Fannounce&tr=udp:%2F%2Fopen.facedatabg.net:6969%2Fannounce&tr=udp:%2F%2Fmgtracker.org:6969%2Fannounce&tr=udp:%2F%2Fipv4.tracker.harry.lu:80%2Fannounce&tr=udp:%2F%2Finferno.demonoid.pw:3418%2Fannounce
Re:Sci Hub repository torrents of scientific papers by mdaniel on Monday May 21, 2018 @11:50AM (#17119249)
I'll try to remember to do it when I get home, but if you beat me to it can you please check and see if that's a known issue with Transmission? Because that seems easy to fix, and in my mind it's bad news to need "vendor specific" magnet links
Re:Sci Hub repository torrents of scientific papers by nneonneo on Monday May 21, 2018 @05:55PM (#17122193)
It sounds like their URL handling might just be terribly broken; see for example https://github.com/transmission/transmission/issues/608.
Re:Sci Hub repository torrents of scientific papers by phyzome on Monday May 21, 2018 @05:04PM (#17121812)
It doesn't look like they have an open issue for that. If you file an issue, you may wish to crosslink it with https://github.com/transmission/transmission/issues/249 which would probably be solved by the same fix for URL query handling.
Re:Sci Hub repository torrents of scientific papers by franzpeterstein on Monday May 21, 2018 @08:06AM (#17117754)
[wget] wget -r -l1 -H -t1 -nd -N -np -A.torrent -erobots=off http://gen.lib.rus.ec/scimag/repository_torrent/
Re:Sci Hub repository torrents of scientific papers by Iv on Monday May 21, 2018 @12:14AM (#17116096)
Is there a way to see which are in the direst need of seeders?

I am currently participating in some of them at random, as I have nowhere near 50 TiB of free space but who love to do it more efficiently.

Re:Sci Hub repository torrents of scientific papers by nneonneo on Monday May 21, 2018 @05:50AM (#17117222)
Possible in theory, just ping the trackers with each of the torrents and see which ones have the fewest peers.
Re:Sci Hub repository torrents of scientific papers by syedkarim on Sunday May 20, 2018 @10:50PM (#17115889)
Even if an entire satellite transponder (36 MHz) was used to broadcast these files (assuming 100Mbps broadcast/downlink), it would still take two months to download the whole thing.
Re:Sci Hub repository torrents of scientific papers by tim-- on Monday May 21, 2018 @04:14AM (#17116885)
Well, that depends on the code rate and modulation.

DVB-S2X 32APSK 32/45 could do 150Mbps with the same bandwidth (as in, the same amount of radio space, not internet bandwidth).

Re:Sci Hub repository torrents of scientific papers by logicallee on Monday May 21, 2018 @02:37AM (#17116572)
???? This is obviously equal to 54,000 datasets of 1 gigabytes each, which I would say is a pretty large dataset to publish along with a paper, so it's hard for me to even imagine 54 thousand such 1-gigabyte datasets. A one gigabyte PDF is astronomical in size, PDF's are usually far far shorter.

So why is this so large? Aren't they just PDF's, basically? And usually just a few pages? Excuse my ignorance. I'm very surprised at the size that you quote (54 TiB).

Re:Sci Hub repository torrents of scientific papers by aldanor on Monday May 21, 2018 @03:24AM (#17116732)
There’s dozens of millions of articles total, not thousands.
Re:Sci Hub repository torrents of scientific papers by logicallee on Monday May 21, 2018 @03:47AM (#17116801)
wow. I didn't know there were a grand total of dozens of millions of journal articles published total worldwide. My guess would have been hundreds of thousands at most. These are journal articles, not high school book reports, right? I'm astounded at the number you quote.

For example there are about 10,000 papers submitted per month to all of arxiv[1] - which is a huge number. That totals 132k per year or so. For there to be "dozens of millions" of journal articles, that would mean arxiv has just 0.83% of them. Given the very low bar to publishing on arXiv I would think if you compared the number of arxiv articles as a percentage of all pubished journal articles, it would be a lot more than 0.83%! So the "dozens of millions" of journal articles is extremely astounding to me.

https://arxiv.org/stats/monthly_submissions

Re:Sci Hub repository torrents of scientific papers by nneonneo on Monday May 21, 2018 @04:08AM (#17116869)
arXiv only handles a small corner of scientific publications, so 0.83% doesn't sound out of whack. Most folks in biology or chemistry aren't on there, for example.

A Nature study from 2014 (http://blogs.nature.com/news/2014/05/global-scientific-outpu...) pegs the number of papers published between 1980 and 2012, with at least one citation, at 38 million. So, 69 million publications including uncited ones and ones from outside this time range actually sounds like an underestimate - that is, the 69 million paper archive is probably missing a fair number of articles.

Re:Sci Hub repository torrents of scientific papers by jacquesm on Monday May 21, 2018 @04:16AM (#17116897)
There are a lot of scientists and they work hard (and get paid peanuts, which seems unfair).
Re:Sci Hub repository torrents of scientific papers by zouhair on Tuesday May 22, 2018 @01:31AM (#17123920)
Unfair and utterly stupid of humanity.
Re:Sci Hub repository torrents of scientific papers by agumonkey on Monday May 21, 2018 @06:58AM (#17117453)
54TiB for publications only ? do you know the total for the books too (which is more libgen than scihub). I remember a 200PiB estimate but I'd love to know more.
Re:Sci Hub repository torrents of scientific papers by teamNOOB on Monday May 21, 2018 @09:33PM (#17123177)
Do you recommend a seedbox for this? I'm currently with Ultraseedbox which is only 3TB. I am thinking of getting a new service specifically to download then provide another source/host of these files.