News for hackers. Stuff that matters.

A comment on Academic Torrents – Making 27TB of research data available.

Re:Academic Torrents – Making 27TB of research data available by leemailll on Sunday August 12, 2018 @09:37AM (#17744295)
I can't find any info on how the data is hosted from the website, so I am wondering whether it works like pirate bay or it also hosts data itself? If the former is the case, it will be hard for researcher to use and share. One reason is that academic institutions nowadays has tightened control on net access, which definitely hinders hosting large amount of data shared with BT protocol; second, because researches are often fragmented which by itself will limit interests of possible users, then the sharing falls on a few people's goodwill.
[ Parent | on HN ]
Re:Academic Torrents – Making 27TB of research data available by epilogue on Sunday August 12, 2018 @10:03AM (#17744385)
It's in the name - Academic 'Torrents'. They just host the torrent files which are only a couple hundred kilobytes. I feel like your sorting of missing the point of a service like this, it's not to provide potentially the fastest download available, but it's to ensure data is accessible, even if the original download source is unavailable or is inaccessible for certain people or locations.

As long as you're not downloading copyrighted data there should be no issue with using the BT protocol on a company or academic network, providing their is no outright ban on the protocol in your network usage policy. The BT protocol itself actually lends itself quite well to large datasets such as what is hosted here due to its inbuilt error checking (so no more spending hours downloading a huge dataset only to find your connection did something silly for a second and corrupted the whole file) and can provide much faster download speeds on popular files due to the number of peers available, instead of a normal hosting arrangement which would likely provide slower speeds on popular files due to network congestion and file access speeds.

[ Parent | on HN ]
Re:Academic Torrents – Making 27TB of research data available by user5994461 on Sunday August 12, 2018 @10:24AM (#17744476)
P2P is banned in the French academic network. I expect the same in neighboring countries, although I didn't get to review their terms of service.
[ Parent | on HN ]
Re:Academic Torrents – Making 27TB of research data available by dewey on Sunday August 12, 2018 @10:42AM (#17744548)
I'd guess if a researcher has to use P2P there's probably a way for them to get the data / get whitelisted. I'm pretty sure the "P2P is banned" is mostly aimed at download copyright infringing content.
[ Parent | on HN ]
Re:Academic Torrents – Making 27TB of research data available by user5994461 on Sunday August 12, 2018 @11:14AM (#17744682)
It's more than copyright infringement. Once you start having a few students or researcher deploying P2P, it's going to saturate the dedicated 10 Gb links very quickly.

P2P will consume any amount of upload bandwidth available. It's horrendous to have inside your network, as a university or research center.

[ Parent | on HN ]
Re:Academic Torrents – Making 27TB of research data available by dewey on Sunday August 12, 2018 @11:18AM (#17744699)
It's not much different than any other service, if you don't limit it or use in moderation it'll saturate the network. If someone would host some linux ISO's over http like some universities do for Linux distributions it'll have the same effect.
[ Parent | on HN ]
Re:Academic Torrents – Making 27TB of research data available by Too on Monday August 13, 2018 @03:02AM (#17748467)
You could limit upload to be inside the network. P2p is not horrendous to have inside your network, it can save you a lot of download-link if people inside the network share files with each other.

Torrents are the most effective, reliable and convenient way to distribute large files, its adoption shouldn't be blocked by bad configuration and policies.

[ Parent | on HN ]
Re:Academic Torrents – Making 27TB of research data available by TomMarius on Sunday August 12, 2018 @10:56AM (#17744608)
Universities are often seeding Linux distro torrents themselves, I'm sure there is a way around that.
[ Parent | on HN ]
Re:Academic Torrents – Making 27TB of research data available by user5994461 on Sunday August 12, 2018 @11:21AM (#17744712)
Linux packages are distributed by HTTP or FTP. There can be public mirrors handled by the network operator or some universities.

It benefits everyone because the automatic mirror selection in Linux distributions picks the lowest latency mirror automatically. That means all OS running in the academic network will pick the public academic mirror, since it's the closest.

[ Parent | on HN ]
Re:Academic Torrents – Making 27TB of research data available by TomMarius on Sunday August 12, 2018 @12:25PM (#17745056)
I'm talking about distro ISOs, not about package manager repositories.
[ Parent | on HN ]
Re:Academic Torrents – Making 27TB of research data available by shellac on Sunday August 12, 2018 @12:43PM (#17745160)
> P2P is banned in the French academic network. I expect the same in neighboring countries

There's no Janet policy (UK academic network) against P2P, although there are institutional policies in place (typically for student residential networks).

To be honest it would be difficult to have a policy that wouldn't impinge on some of the more unusual protocols used in research.

[ Parent | on HN ]
Re:Academic Torrents – Making 27TB of research data available by detaro on Sunday August 12, 2018 @11:23AM (#17744721)
Has never been a problem at my German university (don't know if it is theoretically forbidden or not). Sure, if you misbehaved you'd get in trouble with the sysops, but as long as you don't break the network or do something outsiders loudly complain to the university about nobody cares.
[ Parent | on HN ]
Re:Academic Torrents – Making 27TB of research data available by Fnoord on Sunday August 12, 2018 @10:49AM (#17744575)
"P2P" is a broad term which could include anything federated. Does Tribler [1] work?

[1] https://www.tribler.org

[ Parent | on HN ]
Re:Academic Torrents – Making 27TB of research data available by user5994461 on Sunday August 12, 2018 @11:15AM (#17744686)
They are all banned by the terms of usage. Some of the popular ones are also blocked by technical means.
[ Parent | on HN ]
Re:Academic Torrents – Making 27TB of research data available by dewey on Sunday August 12, 2018 @10:17AM (#17744455)
Based on the sponsors I'd say a lot of the content is hosted by some seedbox companies so you wouldn't have to worry about people seeding at the beginning or on slow connections that much.
[ Parent | on HN ]
Re:Academic Torrents – Making 27TB of research data available by ieee8023 on Sunday August 12, 2018 @04:00PM (#17746139)
The data is hosted by the community and we also coordinate hosting from our sponsors (Listed on the home page)

We work with academic institutions to ensure they allow this service. Please report universities which block the service using the feedback button shown in the lower right of the webpage.

We also encourage HTTP seeds to be specified (aka url-lists) by the uploader to offer a backup URL which can be contacted automatically if BT is blocked. We also offer a python API designed for clusters and university computers written in pure python which supports HTTP seeds: https://github.com/AcademicTorrents/python-r-api

[ Parent | on HN ]
Re:Academic Torrents – Making 27TB of research data available by kankroc on Sunday August 12, 2018 @10:04AM (#17744395)
The name is academic torrents, I can assure you that this is P2P.
[ Parent | on HN ]