News for hackers. Stuff that matters.
Re:Introducing Cloudflare’s IPFS Gateway by orliesaurus on Monday September 17, 2018 @02:12PM (#18008398)
Total noob questions, would love some answers if anyone has them.

Here it says:

"IPFS is a peer-to-peer file system composed of thousands of computers around the world, each of which stores files on behalf of the network. These files can be anything: cat pictures, 3D models, or even entire websites"

- If I run an IPFS node, will I be hosting other people's data? What if these files are illegal? How can my "computer" (I guess a node is the terminology but whatever) be sure I won't be hosting some bad stuff?

Then it says:

"The content with a hash of QmXnnyufdzAWL5CqZ2RnSNgPbvCc1ALT73s6epPrRnZ1Xy could be stored on dozens of nodes, so if one node that was caching that content goes down, the network will just look for the content on another node."

- How long does it take for 1 specific file to be "re-distributed" on another node? and how many times is a file rehosted/distributed on a different node? It would be stupid to have the same file hosted 1.000.000 times - what's the break off point? Is there like a healthcheck so to speak which ensures that a certain file is in danger of disappearing from the network and therefore is automatically pushed for re-distribution

And ultimately, how can I know for sure that CloudFlare won't play the game where as acting as a proxy they will modify some of the files served? Imagine I want to retrieve cat.gif and cloudflare intercepts my request and serves me cat1.gif - I guess it all boils down to trusting them? But hold up, isn't p2p file system like this all about trusting the network and not 1 server?

Re:Introducing Cloudflare’s IPFS Gateway by diggan on Monday September 17, 2018 @02:19PM (#18008474)
The IPFS network works in a pull-model, not push. So when nodes add content, they only add it locally. It's not until some other nodes request it, as it actually transfers.

So if you start a IPFS node locally, it won't get content pushed to it (well, some DHT data which is basically routing information, but not the content itself).

> ultimately, how can I know for sure that CloudFlare won't play the game where as acting as a proxy they will modify some of the files served?

You'll have to add some checks on your end for this as HTTP/browsers cannot provide these guarantees currently (maybe in the future?). So, download file A and add it to IPFS from your end. If you end up with the same hash, the content was not modified.

> isn't p2p file system like this all about trusting the network and not 1 server

Yes, IPFS <> IPFS is trustless as this verification happens automatically and the clients can verify content locally when it gets fetched. IPFS <> HTTP is harder as there is no functionality for doing this verification.

Re:Introducing Cloudflare’s IPFS Gateway by deckar01 on Monday September 17, 2018 @02:27PM (#18008555)
IPFS is a protocol that dictates how it's clients are supposed to validate the data. If you use any of the client libraries they will do this for you (go-ipfs/js-ipfs). If you want to use cloudflare to distribute your index, you need to trust them to use a valid client implementation. Otherwise you can serve an index file off of your own server/network and distribute the rest of the assets via IPFS.
Re:Introducing Cloudflare’s IPFS Gateway by the8472 on Monday September 17, 2018 @03:25PM (#18009135)
> You'll have to add some checks on your end for this as HTTP/browsers cannot provide these guarantees currently

Browser extensions could by verifying content as it arrives.

https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/Web...

Re:Introducing Cloudflare’s IPFS Gateway by AgentME on Monday September 17, 2018 @04:49PM (#18009942)
A browser extension could be used to request the data straight from ipfs instead of Cloudflare.
Re:Introducing Cloudflare’s IPFS Gateway by akavel on Monday September 17, 2018 @05:03PM (#18010069)
Re:Introducing Cloudflare’s IPFS Gateway by bren2013 on Monday September 17, 2018 @05:04PM (#18010074)
I believe the IPFS team has already built a browser extension that runs a js-ipfs node. Running an IPFS node is not ideal for power and bandwidth constrained devices like a mobile phone. This is where I see gateways fitting in, long-term. Not having to trust a gateway unconditionally is necessary to make this viable.
Re:Introducing Cloudflare’s IPFS Gateway by furi on Monday September 17, 2018 @02:26PM (#18008548)
> And ultimately, how can I know for sure that CloudFlare won't play the game where as acting as a proxy they will modify some of the files served? Imagine I want to retrieve cat.gif and cloudflare intercepts my request and serves me cat1.gif - I guess it all boils down to trusting them?

The hash serves that purpose, the file is identified by the hash for the purposes of retrieval and coming up with another file with the same hash (especially one that's usefully different, like changing the overall message in somebody's blog post) is computationally infeasible. Now you might have a problem there in that in a web only implementation like this the code that checks the hash is probably also provided by Cloudflare, but the functionality is there, you can download the file, compute the hash and compare it against the one you retrieved it from.

> But hold up, isn't p2p file system like this all about trusting the network and not 1 server?

Ideally yes but 1. it's very difficult to get more than a handful of users to use anything that doesn't run in a web browser, 2. IPFS' implementation for Windows is effectively unusable (it's a command line interface like the Linux one, which is a non-starter on Windows), 3. attempts to built a complete IPFS node in the browser are currently incomplete, I believe because DHT discovery isn't possible in the browser presently. These gateways should probably be viewed as a stop-gap solution until such a time as we can have full IPFS nodes in the browser.

Re:Introducing Cloudflare’s IPFS Gateway by grittygrease on Monday September 17, 2018 @02:28PM (#18008574)
This browser extension can help you guarantee e2e integrity in the meantime: https://blog.cloudflare.com/e2e-integrity/
Re:Introducing Cloudflare’s IPFS Gateway by bren2013 on Monday September 17, 2018 @02:47PM (#18008801)
> And ultimately, how can I know for sure that CloudFlare won't play the game where as acting as a proxy they will modify some of the files served?

https://blog.cloudflare.com/e2e-integrity/

Re:Introducing Cloudflare’s IPFS Gateway by acover on Monday September 17, 2018 @02:20PM (#18008492)
It's like a torrent. You don't host data that you haven't accessed. For content to stay hosted on your computer forever you have to explicitly pin it.

The number of people hosting it is the number of people accessing it.

CloudFlare can't modify the files. On IPFS you request files by a hash. If they give you a file with a different hash you know they gave you the wrong file.

Re:Introducing Cloudflare’s IPFS Gateway by niij on Monday September 17, 2018 @02:23PM (#18008512)
I think the point is that they're hosting an HTTP proxy to IPFS. Since you're accessing IPFS files over their proxy they can modify the response you receive, since at that point it's just like any other website.
Re:Introducing Cloudflare’s IPFS Gateway by asdkhadsj on Monday September 17, 2018 @02:50PM (#18008826)
> Since you're accessing IPFS files over their proxy they can modify the response you receive, since at that point it's just like any other website.

Correct, but gateways aren't intended to be final solutions. They're compromises targeted at increasing adoption. You wouldn't try and reinvent the internet - you'd make the interoperable where possible, such that adopters can still send normal links, etc.

Re:Introducing Cloudflare’s IPFS Gateway by oscargrouch on Monday September 17, 2018 @02:31PM (#18008613)
> - If I run an IPFS node, will I be hosting other people's data? What if these files are illegal? How can my "computer" (I guess a node is the terminology but whatever) be sure I won't be hosting some bad stuff?

Thats some of the reasons why i prefer the 'Dat' and torrent(be it classic or DHT) model of distribution per content, instead of IPFS which is per block.

The IPFS model make a lot of sense for cloud storage.. You might create a whole CDN based on its model where the clients dont even need to care about your backend storage layer.

But in the per-content p2p distribution model, we have the 'page-rank' effect, where popular or important stuff, curated by human reasoning of whats popular and important.

This model has worked pretty well with torrent, and public interest should drive the p2p storage model, where by using blocks, you lose some of the control and the context. And to see how this is a problem, the IPFS folks created the Filecoin to deal with the problem of the incentives, which on the 'per-content' model of Dat and Torrent, its a non-issue because of the natural page ranking indexing of content drive by public interest.

If something doesnt matter much, it will go away with time, just like our human brain works.

(Of course, if IPFS add some form of indexing of content by public interest on top of the block system, and even let you control what you serve. That problem would be solved)

Re:Introducing Cloudflare’s IPFS Gateway by diggan on Monday September 17, 2018 @02:39PM (#18008700)
(Disclaimer: I work for Protocol Labs on IPFS and other projects [not Filecoin though])

I'm not sure I understand what's different in the way "seeding" or "hosting" of data in Dat and Torrents compared to IPFS. All those requires you to initiate the transfer, nothing gets transferred by itself (unlike FreeNet for example).

Blocks in IPFS is just a detail that a file bigger than X MB gets split up into many smaller "files" so they can be reused between files and simpler to transfer. It has nothing to do with which peers are hosting the content.

In the end, in IPFS, Dat and Torrents, you download and share content based on a ID, and that ID can mean an entire archive of content, or just one picture.

Re:Introducing Cloudflare’s IPFS Gateway by oscargrouch on Monday September 17, 2018 @02:50PM (#18008831)
Its not the technical part. Overall i even think that technically, IPFS has a superior model. I understand it works, and how it works.

What im saying is the control of whats important in a organic form is no there yet. The protocol controls what blocks the peers serve and people dont have much of a saying into it.

But given IPFS its layered in a good design, this could be changed and the block scheduling/routing could be binded more to this organic page-rank by letting people control more which blocks they serve, being more content aware.

This could be even done in a topic level, say you are ok to serve "free software". I guess that if you guys implemented something like that, giving some control to the peers of what they serve, it will be much more easy to sell the concept to everybody.

Re:Introducing Cloudflare’s IPFS Gateway by oscargrouch on Monday September 17, 2018 @03:13PM (#18009025)
Edit: Nevermind my parent comment, i guess the 'pinning' concept of IPFS does what im trying to elucidate here.. sorry for the noise.

(But the block scheduling algorithm could/might change in the future maybe? turining this into a real issue?)