News for hackers. Stuff that matters.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by grumbel on Tuesday January 16, 2024 @09:53AM (#39013875)
Has it? I have never seen anybody using IPFS in the wild, even projects for which it should be well suited (archive.org, Linux package distribution, Git, Lemmy, Imgur, CivitAI), don't use it. Worse yet, IPFS still provides no real way to deal with private or local data, which drastically limits its use.

I love the idea about data being addressable by hash, but I don't feel IPFS has actually delivered anything meaningful in that area yet.

And with ipfs-search.com shutdown, there is not even any way left to explore what is actually on the network now.

And technical issues aside, there is also the legal problem that IPFS conflicts with copyright. Redistributing anything is illegal by default, unless somebody gives you permission, IPFS provides no means to track that permission. You can't attach a GPL or an author to a hash.

Re:Filecoin Foundation Successfully Deploys IPFS in Space by orblivion on Tuesday January 16, 2024 @12:19PM (#39015876)
> Linux package distribution

FWIW it seems like NixOS at least tried or is trying: https://blog.ipfs.tech/2020-09-08-nix-ipfs-milestone-1/

Re:Filecoin Foundation Successfully Deploys IPFS in Space by LanternLight83 on Tuesday January 16, 2024 @02:11PM (#39017526)
Re:Filecoin Foundation Successfully Deploys IPFS in Space by tutfbhuf on Tuesday January 16, 2024 @10:18AM (#39014191)
> Has it? I have never seen anybody using IPFS in the wild

Every time someone downloads a book from a shadow library like Library Genesis, it's through IPFS most of the time, often via an IPFS gateway such as Cloudflare's IPFS gateway, so you don't even notice it's using IPFS. These shadow libraries have millions of users per day, especially academics.

Re:Filecoin Foundation Successfully Deploys IPFS in Space by shuckles on Tuesday January 16, 2024 @10:47AM (#39014635)
I’ve recently observed that both IPFS and the Cloudflare gateways are extremely unreliable on Libgen.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by poopsmithe on Tuesday January 16, 2024 @12:51PM (#39016305)
That's besides the point. The point being made was that IPFS is being used in production.

You bring up a valid point though. From my experience, IPFS suffers from a tragedy of the commons issue. It's performance is generally terrible because the free providers are overloaded.

Re:Filecoin Foundation Successfully Deploys IPFS in Space by shuckles on Tuesday January 16, 2024 @01:22PM (#39016660)
Not really:

> Every time someone downloads a book from a shadow library like Library Genesis, it's through IPFS most of the time

I would like to see evidence this is actually true. My theory is that IPFS's "success" in hosting Libgen is roughly turning ICO proceeds into free hosting. Paying for dedicated servers might have been more cost efficient and, given the performance issues, more robust!

Re:Filecoin Foundation Successfully Deploys IPFS in Space by beaugunderson on Tuesday January 16, 2024 @04:17PM (#39019255)
100%, and same for Standard Template Construct. There are basically no reliable IPFS gateways with book or paper content. They all get struck from the more reliable gateways immediately.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by shuckles on Tuesday January 16, 2024 @07:03PM (#39021066)
Unfortunate behavior in an apocalypse-proof distributed filesystem.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by grotorea on Tuesday January 16, 2024 @04:09PM (#39019177)
So what we already knew, IPFS is a slightly better torrent.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by anacrolix on Thursday January 18, 2024 @08:17AM (#39041302)
It's a significantly worse BitTorrent. BitTorrent is fantastic.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by rglullis on Tuesday January 16, 2024 @10:12AM (#39014109)
I was working on the same building as the team from Protocol Labs. I talked with a couple of devs and it seems like they never even considered that most people who would be interested in running a node would like to have control over who gets access to the pinned files. I think I opened a ticket asking for an ACL system, but it got closed.

I really had high hopes for it, but I realized that all I really want is an object storage with content addressable urls.

Re:Filecoin Foundation Successfully Deploys IPFS in Space by pclmulqdq on Tuesday January 16, 2024 @10:16AM (#39014163)
ACLs are fundamentally incompatible with blockchains. Your only solution is to encrypt those files, and have off-chain key management if you ever want to revoke access.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by simpsond on Tuesday January 16, 2024 @10:28AM (#39014332)
ACLs are compatible with some blockchains. Token gating, merkle proofs, and RBAC are fairly common in smart contracts. See: https://docs.openzeppelin.com/contracts/2.x/access-control#r...
Re:Filecoin Foundation Successfully Deploys IPFS in Space by pclmulqdq on Tuesday January 16, 2024 @03:48PM (#39018921)
In terms of interacting with smart contracts, yes. In terms of accessing data, no. All data on a blockchain is public and immutable, and that's the point.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by rglullis on Tuesday January 16, 2024 @10:16AM (#39014167)
IPFS and distributed filesystems have nothing to do with blockchains.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by mhluongo on Tuesday January 16, 2024 @12:23PM (#39015928)
Filecoin, the thing funding IPFS, certainly does :)
Re:Filecoin Foundation Successfully Deploys IPFS in Space by dartos on Tuesday January 16, 2024 @12:48PM (#39016262)
Sure, but IPFS, the technology, doesn’t.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by pclmulqdq on Tuesday January 16, 2024 @03:49PM (#39018930)
IPFS is entwined with (and pretty much 100% backed by) Filecoin, which is a blockchain. Introducing a point of incompatibility would be a nonstarter.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by rglullis on Tuesday January 16, 2024 @04:37PM (#39019501)
Nothing is "entwined". I can download kubo and run it without even knowing what Filecoin is. In fact, every user of Brave browser does it. Same thing for projects like OrbitDB. Same thing for customers of Pinata.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by pclmulqdq on Tuesday January 16, 2024 @05:55PM (#39020318)
I can download F2FS and run it on my RAM. That doesn't mean that F2FS isn't entwined with NVMe SSDs.

In fact, while F2FS is a ludicrous example, this happens regularly with filesystems. The core design of filesystems is still built assuming that a block device will be behind it. IPFS is the same way, just s/block device/blockchain/.

Re:Filecoin Foundation Successfully Deploys IPFS in Space by rglullis on Tuesday January 16, 2024 @06:32PM (#39020720)
I'm sorry, I really have no idea where you are trying to go with this.

- You can have files on IPFS and no blockchain whatsoever "behind it".

- IPFS itself makes no use of blockchains as an abstration for data storage or file organization

- If all the blockchains networks and nodes disappeared overnight, no IPFS nodes would be affected.

I don't know if you actually used any of that or just going by hearsay.

Re:Filecoin Foundation Successfully Deploys IPFS in Space by unicornmama on Tuesday January 16, 2024 @11:52AM (#39015492)
But nginx HTTP bridge with copyright banlist is OK…
Re:Filecoin Foundation Successfully Deploys IPFS in Space by fulladder on Tuesday January 16, 2024 @01:35PM (#39016854)
IPFS does not in any way involve a blockchain.

There's a separate "layer on top of IPFS" called Filecoin that does involve a blockchain. (From a technical point of view, it's not really a layer on top, more of a re-implementation of IPFS with a blockchain.)

I've worked on IPFS and know many of the key people at PL. They really are not blockchain people or cryptobros. They are much different from anyone else in crypto I have encountered. But ultimately they have to fund their activities somehow, and this is the reality of where the business potential is in P2P circa mid-2020s.

Re:Filecoin Foundation Successfully Deploys IPFS in Space by dhash on Tuesday January 16, 2024 @12:56PM (#39016374)
Hi, PL-funded founder here.

This was one of the things that made IPFS a non-starter for us. We ended up grafting Hashicorp Vault into kubo (the go-ipfs implementation) so that we could use IPFS and have things like detetes and access revocation that actually work.

Re:Filecoin Foundation Successfully Deploys IPFS in Space by jnthnvctr12 on Tuesday January 16, 2024 @03:37PM (#39018797)
i used to work at PL, surprised you got that response (don't know who you talked to). Meanwhile, folks have been building things like that:

https://guide.fission.codes/developers/webnative/file-system...

Re:Filecoin Foundation Successfully Deploys IPFS in Space by ianopolous on Wednesday January 17, 2024 @06:15AM (#39026257)
We've been pioneering privacy and access control on top of IPFS since 2015 in Peergos [0].

We have block level access control for example: https://peergos.org/posts/bats

As well as E2EE, sharing, trustless servers and more [1].

[0] https://github.com/peergos/peergos

[1] https://book.peergos.org

Re:Filecoin Foundation Successfully Deploys IPFS in Space by rglullis on Wednesday January 17, 2024 @06:45AM (#39026471)
Ok, that is really cool. I worry though that this is too tied into the whole peergos architecture? Can you give some pointers on what would I need to do to have a node with the extended protocol and nothing else?
Re:Filecoin Foundation Successfully Deploys IPFS in Space by ianopolous on Wednesday January 17, 2024 @12:49PM (#39030985)
The block auth is very generic, just an extra top level key in cbor maps or a small prefix in raw blocks. The protocol is S3 V4 signatures.

There is a Java implementation of the modified bitswap for this in Nabu [0] and a Go one in ipfs-nucleus [1]. With both of these you can use whatever auth verification protocol you like.

[0] https://github.com/peergos/nabu

[1] https://github.com/peergos/ipfs-nucleus/

Re:Filecoin Foundation Successfully Deploys IPFS in Space by jrwr on Tuesday January 16, 2024 @10:11AM (#39014094)
IPFS also has issues with large datasets, anything larger then a few TBs cases nodes to explode and indexing takes forever.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by realPubkey on Tuesday January 16, 2024 @10:41AM (#39014524)
Also the lookup time for files is too slow to use it on anything user facing.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by geokon on Tuesday January 16, 2024 @10:51AM (#39014692)
Wouldn't all those examples also work just as easily with torrents/magnet-links? I think in all those cases a central server distribution model has unfortunately been "good enough" for the majority of users (even though their data is mined and ads are injected)

When it comes to FOSS, I personally don't understand why something like a package manager isn't P2P by default. It feels very aligned with the hacker culture - a la "A Declaration of the Independence of Cyberspace". Virtually nobody uses it, so the solutions are half baked, clunky and not integrated with every day workflows (ex: browsers don't support anything P2P). Something like libcurl can't pull a torrent from the web

Re:Filecoin Foundation Successfully Deploys IPFS in Space by sseagull on Tuesday January 16, 2024 @12:48PM (#39016261)
> Wouldn't all those examples also work just as easily with torrents/magnet-links?

My (possibly wrong) feeling is that IPFS is meant for smaller files, and torrents for bigger collections. So with the example of a package manager, you could download each individual package via IPFS, whereas a torrent would make sense if you wanted to download an archive of them all.

Not to say torrents couldn't work, but it just kinda feels like not the intended use case.

(The above feeling come from reading about IPFS a few times throughout the years and toying with it for a while).

Re:Filecoin Foundation Successfully Deploys IPFS in Space by anthk on Tuesday January 16, 2024 @03:26PM (#39018645)
P2P works better for ISO based distros with full offline backups or maybe LTS based distros with small repos as Hyperbola. Having a full repo of Trisquel with continuous updates would not work well.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by geokon on Tuesday January 16, 2024 @11:52PM (#39023458)
Oh right, that's a very good point. I've only use LTS for years. Forgot about rolling releases haha
Re:Filecoin Foundation Successfully Deploys IPFS in Space by rakoo on Wednesday January 17, 2024 @06:43AM (#39026449)
There has been attempts at doing torrent-based package distribution, see the notes of debian testing it here: https://wiki.debian.org/DebTorrent. There are basically 2 problems:

Packages are a lot of small files, and not everyone is interested in them all. Torrents are good for large content that everyone wants. The two are in opposition and there needs to be a balance: do you need to make One Unique Torrent with everything, but then peers have to individually ask each and every other peer if they have the content they're looking for, making it unscalable ? Or do you make one torrent per package, meaning peers have to track thousands of torrents ? (That's the direction IPFS took, which is why it's extremely inefficient and resource-heavy). DebTorrent has made the choice to go in the middle.

The real problem, to me, is that content-addressed content is the anti-privacy protocol: since you ask everyone "do you have this content", you're basically telling everyone what you want. In the case of installing packages, this means telling everyone what the version of your packages is, and is potentially a security issue.

But the real "problem" is that it fails at being much more better than the existing solution. Bandwidth is cheap, storage is cheap, so hosting everything on a few independent but centralized servers is good enough.

Re:Filecoin Foundation Successfully Deploys IPFS in Space by rvz on Tuesday January 16, 2024 @12:17PM (#39015844)
> I have never seen anybody using IPFS in the wild, even projects for which it should be well suited (archive.org, Linux package distribution, Git, Lemmy, Imgur, CivitAI), don't use it.

Netflix uses IPFS. [0]

[0] https://blog.ipfs.tech/2020-02-14-improved-bitswap-for-conta...

Re:Filecoin Foundation Successfully Deploys IPFS in Space by diggan on Tuesday January 16, 2024 @10:08AM (#39014058)
> there is also the legal problem that IPFS conflicts with copyright. Redistributing anything is illegal by default

I think you might misunderstand how IPFS works, or possibly confuse it with Freenet. When you use IPFS, nothing is automatically distributed, unless they happen to know the hash of the content you've added locally. It's not until someone starts to explicitly request data from you, that you start sharing anything.

Re:Filecoin Foundation Successfully Deploys IPFS in Space by grumbel on Tuesday January 16, 2024 @10:12AM (#39014104)
Everything in your local cache is redistributed, meaning every single website you view through IPFS ends up getting potentially redistributed. Can you guarantee that each and every item you view on the Web allows redistribution?
Re:Filecoin Foundation Successfully Deploys IPFS in Space by jerbear4328 on Tuesday January 16, 2024 @10:21AM (#39014224)
I don't know how it works legally, but if you publish your own content to a distributed protocol like IPFS you should expect it to be distributed. If content that shouldn't be redistributed is put on the public IPFS, it should be the person who added it's fault, imo.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by grumbel on Tuesday January 16, 2024 @10:27AM (#39014310)
The person who added it, is not tracked. A file can be added by anybody at anytime and anywhere and it will get the same content address. It's up to the person doing the redistribution to ensure it's legal, but IPFS gives them no tools to do so.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by amelius on Tuesday January 16, 2024 @12:50PM (#39016282)
Perhaps it should use the fact that any file A can be written as the XOR of two files B and C, where B and C look like complete noise. If you only host B and another person only hosts C, then there can be no copyright infringements, while you can still reconstruct A by simply XORing the other files together. Of course at the expense of having to store and transfer twice the amount of data.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by fulladder on Tuesday January 16, 2024 @01:44PM (#39017023)
This is engineer logic. This kind of thing would never fly in court. If you want an example, go back and look at what happened to Aereo back in 2014. They came up with a clever hack to work around broadcast TV copyright. It went to the U.S. Supreme Court and got shot down.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by amelius on Tuesday January 16, 2024 @03:56PM (#39019024)
I don't think so.

Assume A is the copyrighted work. Now construct a file of random bits and call it B. Then assuming A=B^C, then you can solve C=A^B (where ^ is the XOR operator). Both B and C appear to be completely random.

Assume you are sued because C is found on your harddrive and if you XOR it with B (found on someone else's harddrive) you get copyrighted work A.

To construct your defense, take a work in the public domain, and call it P. Compute D=C^P. Now tell the judge that there is a file D on a friend's harddrive, that allows you to reconstruct P.

Alternatively, tell the judge that you are researching random numbers and C is just a file you work with. It is not illegal to have random numbers (bits) on your harddrive.

Re:Filecoin Foundation Successfully Deploys IPFS in Space by Dylan16807 on Tuesday January 16, 2024 @04:49PM (#39019625)
I see it pretty differently.

I mean, the XOR thing is a bad idea, because obviously at least one of those files is violating copyright. Trying to feign innocence is not engineer logic, it's failed logic.

But Aereo I'm still mad about. They saw that distribution is what cable companies have to pay lots of money for, so they removed distribution. It should have been fine.

The effect of the decision is that it's legal for a person to set up a remote antenna for personal use, but it's a copyright infringement to pay a company to set it up for you. Or, perhaps, that it's possible to pay a company to do so but only if it's your idea as a one-off. Either way, that doesn't make any sense as a copyright issue.

And after the supreme court decision, they tried to pivot to legally being a cable service... only to get denied recognition as one!

Re:Filecoin Foundation Successfully Deploys IPFS in Space by fulladder on Tuesday January 16, 2024 @05:37PM (#39020126)
I'm sort of mad about Aereo too. My view is the same as yours. It's unfortunate that it got shut down.

My point in the GP comment was more that if the entire justification for your technical architecture is to make an end run around some law, you have to consider the possibility that a judge is going to see what you're trying to do and may have a different interpretation. There's a big human element in the law that I think engineers, especially "code as law" blockchain people, often overlook or misunderstand.

Maybe in the future we'll have AI judges and AI lawyers and your case will be decided in 100 milliseconds. However, that's not what we have today.

Re:Filecoin Foundation Successfully Deploys IPFS in Space by amelius on Tuesday January 16, 2024 @06:22PM (#39020615)
> There's a big human element in the law that I think engineers, especially "code as law" blockchain people, often overlook or misunderstand.

The point with the XORing is that you can't blame one person for the infringement.

Imagine twin brothers. One of them steals a painting and is caught on camera. The judge knows that one of the twins is guilty. However, he can't convict either of them exactly because of the human element in the law.

Re:Filecoin Foundation Successfully Deploys IPFS in Space by sandbx on Tuesday January 16, 2024 @10:06AM (#39014029)
libgen uses it
Re:Filecoin Foundation Successfully Deploys IPFS in Space by timtom39 on Tuesday January 16, 2024 @10:07AM (#39014042)
In practice libgen uses torrents. The available seed nodes for IPFS are... few. This is largely due to the software/protocol being pretty bad compared to battle tested torrents.
Re:Filecoin Foundation Successfully Deploys IPFS in Space by arsome on Tuesday January 16, 2024 @11:36AM (#39015250)
Perhaps if you're downloading full dumps, but distribution of individual books is often done via IPFS.