News for hackers. Stuff that matters.
Re:Dat: non-profit, secure, and distributed package manager ... by filiwickers on Tuesday September 26, 2017 @08:35PM (#15343919)
Hi everyone, I'm one of the core contributors to Dat, @joeahand. Happy to answer any questions. It's an interesting time to see this posted because I've been working on a new datproject.org site recently =).

Dat Project started with a focus on increasing access to research & public data. To support the data tools we built a peer-to-peer protocol. People are doing some really cool stuff on top of Dat (such as Beaker Browser), we're really excited about it and want to make sure to support all the neat use cases.

We'll be launching an updated site soon to highlight more of the work around the protocol and what the community is building. Our main use case will still be data management but most of what you see on the current site will shift to a new domain.

Re:Dat: non-profit, secure, and distributed package manager ... by baldfat on Wednesday September 27, 2017 @07:39AM (#15346752)
Care to explain how this is different then Resilio https://www.resilio.com/

I use this with encryption for my data folders on my projects.

Re:Dat: non-profit, secure, and distributed package manager ... by filiwickers on Wednesday September 27, 2017 @11:24AM (#15348713)
Ya, there are a few other related questions below. Resilio is BitTorrent-based. But I'm not 100% familiar with how Resilio differs from BitTorrent.

The core difference is in our approach. We're all open source and a non-profit. We're also really focused on the research data use case where BitTorrent is less easily deployed.

We hope that making an open and easy to use p2p protocol will enable other developers to build applications on top, and something like Resilio could be one.

Re:Dat: non-profit, secure, and distributed package manager ... by rakoo on Wednesday September 27, 2017 @04:35AM (#15345893)
I've glanced at the docs before already, and dat seemed super cool except for one little thing: it seems that there can only be one source that is allowed to modify the dataset. Is this by design or am I missing something ?
Re:Dat: non-profit, secure, and distributed package manager ... by tbv on Wednesday September 27, 2017 @08:54AM (#15347268)
That’s correct. But the protocol author mafintosh is currently working on a major new release which introduces multi-writer capabilities.
Re:Dat: non-profit, secure, and distributed package manager ... by molszanski on Wednesday September 27, 2017 @09:23AM (#15347527)
It looks like to me, from the sources, that you see plain http/tcp connection. BitTorrent is using UDP that could be a huge gain in many situations. Do you plan to add some Realiable UDP overlay to speed things up? Something like Aspera but more open? Maybe QUIC is mature enough?
Re:Dat: non-profit, secure, and distributed package manager ... by filiwickers on Wednesday September 27, 2017 @11:19AM (#15348638)
We use TCP or UDP for all the connections. UDP is especially helpful for hole-punching as well.

The direct HTTP support is not fully supported yet. But we're excited for that because it'll allow you to use S3 or other static file servers as peers.

Dat works over any protocol, so it's just a matter of implementing it.

Re:Dat: non-profit, secure, and distributed package manager ... by molszanski on Thursday September 28, 2017 @07:54AM (#15356420)
Thanks! I usseme i isimplemented in mafintosh/hyperdrive?
Re:Dat: non-profit, secure, and distributed package manager ... by batbomb on Wednesday September 27, 2017 @12:38AM (#15344966)
How are you going to beat Globus for research data of appreciable size?
Re:Dat: non-profit, secure, and distributed package manager ... by filiwickers on Wednesday September 27, 2017 @11:29AM (#15348762)
We'd love to work with them! We really like what they've done and we have a few partners that use Globus.

Underneath all the transfers in Globus use GridFTP. Using Dat could help distribute bandwidth and speed up transfers. It'll also add version control for free, which (I think) Globus does not have yet.