News for hackers. Stuff that matters.
Re:IPFS, The Interplanetary File System, Simply Explained by mStreamTeam on Monday December 10, 2018 @02:53PM (#18650670)
Can someone explain to me how IPFS is superior to BitTorrent from a usability perspective?

I get all the additional functionality IPFS comes with, but it just feels too complex to get adopted. I can go out right now and start interacting with BitTorrent networks, but I don't even know where to get started with an IPFS system

Re:IPFS, The Interplanetary File System, Simply Explained by espadrine on Monday December 10, 2018 @03:18PM (#18650866)
BitTorrent started out without a DHT, and so requires metadata and a torrent file to start downloading.

IPFS has a unique, global DHT as a matter of principle, and so, only requires the hash of the file to start downloading. Everyone gets the same bootstrap nodes, and then is part of the same DHT, with a common lookup mechanism relying on Kademlia.

On the other hand, I feel like IPFS is not as advanced as dat in relevant ways. IPFS is not good for file edition, while dat has systems that simplify modification of even large files and merging of structured files.

Re:IPFS, The Interplanetary File System, Simply Explained by antocv on Monday December 10, 2018 @04:01PM (#18651138)
Bittorrent transport layer, peer-to-peer is not encrypted, neither was IPFS last time I checked. Is it now? Has that changed?

There is nothing stopping Bittorrent clients from agreeing on "this dht will be global", and similary, what stops IPFS clients/peers from forming a different DHT? Nothing as far as I know, not like scuttlebutt which has a transport-layer network-wide key.

What about trackers, bittorrent can but does not work well without trackers, so its not really according to me, decentralized, since some peer is more equal than another peer.

IPFS is meh, datproject is where its at.

Re:IPFS, The Interplanetary File System, Simply Explained by kaffee on Monday December 10, 2018 @04:20PM (#18651250)
IPFS has perhaps 2,000 times the resources ($) of dat. (Raised via the filecoin sale.) Surely that must count for a little.
Re:IPFS, The Interplanetary File System, Simply Explained by antocv on Monday December 10, 2018 @04:45PM (#18651416)
Considering how much resources, github starts, and fanbois IPFS has, and yet the network layer is still shit, bandwidth not utilized to fullest.

Instead of just reusing bittorrents piece-per-peer-exchange-protocol (that greedy not perfect algorithm, forgot its name) - which can saturate even high end links, they invented bitswap, full with bugs, then decided lets make filecoin instead, like a bartering engine of exchanging pieces.

datproject also can saturate my 250mbit connection.

Re:IPFS, The Interplanetary File System, Simply Explained by mStreamTeam on Monday December 10, 2018 @05:00PM (#18651517)
Thanks for posting. Never heard of DAT before but it looks to be exactly the BitTorrent alternative I've been looking for
Re:IPFS, The Interplanetary File System, Simply Explained by macawfish on Monday December 10, 2018 @06:56PM (#18652214)
Check out hyperdb and hyperdrive (implementations of the core replicable data structures used in dat)!
Re:IPFS, The Interplanetary File System, Simply Explained by macawfish on Monday December 10, 2018 @06:13PM (#18651955)
One more note: dat's implementation is quite modular, the hyper* libraries are designed to be replicated in a network-agnostic way.

(Theoretically you can do this with IPFS too, but I love how dat is implemented!)

Re:IPFS, The Interplanetary File System, Simply Explained by colemickens on Monday December 10, 2018 @09:46PM (#18652987)
I'm trying to get a foothold in this space (this space being whatever the area of camlistore/perkeep - ipfs - dat - etc.

I've also been excited that Dat seems to be investing a Rust implementation that would presumably make Dat accessible from any language that can interoperate with C/Rust.

Re:IPFS, The Interplanetary File System, Simply Explained by macawfish on Wednesday December 12, 2018 @06:21AM (#18662952)
Yeah, I'm keeping an eye on the rust implementation of hyperdb!
Re:IPFS, The Interplanetary File System, Simply Explained by brighton36 on Monday December 10, 2018 @04:59PM (#18651503)
It might make it count for less, considering the liability it now has to make investors whole
Re:IPFS, The Interplanetary File System, Simply Explained by ric2b on Wednesday December 12, 2018 @11:06AM (#18665012)
It was an ICO, there are no such liabilities.
Re:IPFS, The Interplanetary File System, Simply Explained by willio58 on Monday December 10, 2018 @04:23PM (#18651270)
It should, but it might end up not.
Re:IPFS, The Interplanetary File System, Simply Explained by zaarn on Tuesday December 11, 2018 @01:53AM (#18653623)
BitTorrent has Infohashes which allow you to download a torrent merely by entering the hash (not to be confused with magnet links, which may also be just pointing towards the infohash)
Re:IPFS, The Interplanetary File System, Simply Explained by ntninja on Tuesday December 11, 2018 @03:50PM (#18658725)
True, but one big caveat with BitTorrent is that if you change 1 byte in the whole (often multi-file) thing the info-hash will be different and you'll end up in a new DHT entry all by yourself. With IPFS you'd still share all the other blocks of content with the whole network, only the modified block will be “yours alone”.

Where this happens to be particularly annoying with BitTorrent are aggregate Torrents: You have a bunch of separate files available for download (think TV series or books) each forming its own DHT node. Then somebody decides to create an aggregate torrent (such as libgen's “1000 books” torrents) and offers that for convenience or efficiency reasons. Will the clients that previously downloaded the individual files share them with this new torrent? No! Because the new Torrent “is different” even if each of it's files are byte-by-byte identical.

Could BitTorrent be upgraded to allow for this? Probably, but it'd be radical paradigm-shift requiring a redesign of everything (“BitTorrent 2.0”). So why not start from scratch with a new protocol entirely?

Re:IPFS, The Interplanetary File System, Simply Explained by zaarn on Tuesday December 11, 2018 @03:57PM (#18658776)
There are actually several protocol updates in the works (and some fundamental work like pubsub has been finished) to enable torrents that can be upgraded, it doesn't require a radical paradigm-shift at all.

A protocol that does what torrent does but better would be dat://, IPFS is terrible at being BitTorrent but better.

Re:IPFS, The Interplanetary File System, Simply Explained by jchanimal on Monday December 10, 2018 @06:42PM (#18652135)
I've started using the `ipfs add` command to share files that are too big to email. The users can just download the file from a gateway like ipfs.io without installing anything, and the file is only hosted on my laptop.
Re:IPFS, The Interplanetary File System, Simply Explained by jstanley on Monday December 10, 2018 @03:31PM (#18650957)
This should get you started: https://docs.ipfs.io/introduction/usage/

It's quick and easy. I recommend the browser extensions.

Re:IPFS, The Interplanetary File System, Simply Explained by acob on Monday December 10, 2018 @03:33PM (#18650976)
Excellent resource, thank you for sharing.
Re:IPFS, The Interplanetary File System, Simply Explained by dana321 on Monday December 10, 2018 @03:13PM (#18650838)
https://d.tube is probably the best open example of ipfs thats fairly easy to use.

Otherwise, its first and foremost for geeks, then other geeks implement apps for the normals using the technology.

Re:IPFS, The Interplanetary File System, Simply Explained by rgardaphe on Tuesday December 11, 2018 @12:16PM (#18656820)
We (qri) have built a front-end webapp (free & open source) for publishing, sharing, and versioning datasets on IPFS. When you download qri, you're working on IPFS (distributed web).

Still very geeky, but we're trying to make sure it's very simple to use for data scientists/analysts/researchers alike.

here's more on us: https://qri.io & https://github.com/qri-io

Re:IPFS, The Interplanetary File System, Simply Explained by acob on Monday December 10, 2018 @03:18PM (#18650860)
D Tube actually runs on top of Steem Blockchain and uses IPFS as a Filesystem.

This is similar to how Tron will work.

Re:IPFS, The Interplanetary File System, Simply Explained by dana321 on Monday December 10, 2018 @04:49PM (#18651446)
I know that, you don't need steem to view videos on the site, so its a good accessible example of an easy use site that uses ipfs.
Re:IPFS, The Interplanetary File System, Simply Explained by acob on Monday December 10, 2018 @03:18PM (#18650864)
Re:IPFS, The Interplanetary File System, Simply Explained by StavrosK on Monday December 10, 2018 @03:25PM (#18650918)
IPFS is like if BitTorrent had a single swarm and the hash chose a set of files to download from that.
Re:IPFS, The Interplanetary File System, Simply Explained by user2426679 on Monday December 10, 2018 @03:24PM (#18650907)
They're not mutually exclusive. There is a popular private tracker that uses IPFS to host screenshot previews, while serving the actual videos via torrent.
Re:IPFS, The Interplanetary File System, Simply Explained by perfmode on Monday December 10, 2018 @02:54PM (#18650681)
There is a single global swarm.
Re:IPFS, The Interplanetary File System, Simply Explained by silotis on Monday December 10, 2018 @03:26PM (#18650922)
I'm much more familiar with Bittorrent than IPFS, but as I understand it IPFS treats each object as its own swarm. This is quite the opposite of having a single global swarm. In Bittorrent terms it's as if each piece and the torrent itself are all separate swarms.

What makes IFPS more global is that they have defined a set of standards for linking objects together to form potentially complex structures. The holy grail is to scale this up to create a whole new World Wide Web based on linked IFPS objects rather than linking to resources on specific hosts.

Re:IPFS, The Interplanetary File System, Simply Explained by setr on Monday December 10, 2018 @04:11PM (#18651188)
Its still not clear to me if its true, but my impression from reading the whitepaper a couple years back was that you could fetch hashblocks for your target file, from any arbitrary node that happens to be serving it, regardless of what the original target was. Eg a block composed purely of white pixels can be equally shared across thousand of gifs, independent of the original file.

Which if true would make it more of a global swarm

Oddly, it would also imply you could potentially generate entirely new files out of the network; in an infinitely large network, you wouldnt even have to specifically upload a new file — the hashtree would be enough, and the network would find the pieces to compose it

Never got anyone to confirm/deny my understanding of it, and its not clear to me why it wouldnt be the case, if it weren’t.

Re:IPFS, The Interplanetary File System, Simply Explained by krrrh on Monday December 10, 2018 @05:02PM (#18651527)
Given that there would never be an infinitely large network, how likely is it that this sort of de-duplication would offer benefits? We use hashes to ensure data integrity because if a single bit is changed in the data the hash will reliably change. GIF compression already reduces "256 white pixels in a row" to a few bytes (other compression algorithms aren't so straightforward). Even with all the white gifs people are sharing online, you'd need vanishingly small block sizes for this strategy to achieve anything measurable, approaching the point where the hash doesn't just represent the block, but is the block.
Re:IPFS, The Interplanetary File System, Simply Explained by setr on Monday December 10, 2018 @05:42PM (#18651760)
It's more of it's not clear to me why you wouldn't implement it in this fashion, my impression from the paper was that it was implemented in this fashion, and as a side-effect of that implementation, this kind of de-duplication would be possible somewhat by accident.

More specifically, its not clear to me what benefit exists to associating a block with its originating file; my understanding of bittorrent requires you to receive/send out the total list of blocks anyways (the .torrent file), as you're requesting the missing blocks (and at start, thats all blocks), and others are responding with blocks the have to offer (which may not be all blocks, since you can request from partially-finished peers aka leechers).

In IPFS, you don't have an associated server (a tracker) to tell you who your peers are for a given file, so first you'll ask around for the list of blocks required for that file, and then you'll ask anyone who responded for your missing blocks, same as bittorrent. But you'll have to keep asking for new peers, since you aren't being updated by the server, and sending out the list of missing blocks all the same.

It seems to me that there's not much benefit to requesting specifically the file, and then the blocks, rather than just requesting the blocks directly, when doing that updated request.

And if you're asking for the blocks directly... then that kind of de-duplication would be a natural outcome.

Also a spot where it might actually be impactful today -- Memes.

And if its truly an interplanetary mindset, there might come a point long into the future where such a feature would be an actual feature, rather than an accident. But regardless, I only mean it as an implication of how I believe IPFS is implemented, not an intended benefit/feature of it. I specifically mean it as an easy way to point out where my understanding failed, if I'm wrong; or an easy way to verify my understanding is correct ;)

Re:IPFS, The Interplanetary File System, Simply Explained by perfmode on Monday December 10, 2018 @06:45PM (#18652158)
I wrote the original data exchange (bitswap) code. Your first paragraph is true.
Re:IPFS, The Interplanetary File System, Simply Explained by perfmode on Monday December 10, 2018 @10:21PM (#18653129)
> Oddly, it would also imply you could potentially generate entirely new files out of the network; in an infinitely large network, you wouldnt even have to specifically upload a new file — the hashtree would be enough, and the network would find the pieces to compose it

It's vanishingly unlikely that this would yield useful results in practice.

In a moderately-sized network, however, if only a few contiguous bytes in a GB file change, then only a single block would need to be uploaded. (if the original file is sufficiently available already)