News for hackers. Stuff that matters.
Re:The Internet Archive and its 916B saved web pages are bac... by Cheer2171 on Wednesday October 16, 2024 @08:26AM (#41858279)
You say this as if it is an original idea. Of course the IA is working on this and have been for over 6 years. There already is a DWeb version. They have been advancing DWeb infrastructure. The IA hosts all kinds of DWeb developer events.

But it is over 50 petabytes and the IA gets a huge amount of traffic through the regular web that they need to serve quickly and efficiently to their users.

Guess what has happened over 6 years of decentralization of 50 TB? People only seed what they want or care about and there aren't enough seeders to host. They set all this up and nobody volunteers. You're a DWeb advocate and you haven't been seeding. That's a recipe for disaster if they rely on the goodness of volunteer seeders. The IA's mission is broader. DWeb will ever only compliment the IAs mission.

https://blog.archive.org/2021/02/18/behind-the-scenes-of-the...

https://www.bleepingcomputer.com/news/technology/archiveorg-...

Re:The Internet Archive and its 916B saved web pages are bac... by ksymph on Wednesday October 16, 2024 @09:04AM (#41858621)
How does one contribute? In the article you linked:

> there is no information on how users can get involved in the decentralized version of Archive.org and who the peers are that are distributing the content.

The other link doesn't mention how people could help host data either. If there is a way, then it seems like more of a marketing issue if those willing are unaware or unable to figure out how. I can't find any actionable steps on how to contribute.

edit - it seems the dweb version was a frontend for archive.org testing serving IA content over alternative protocols. It was never finished or expanded on unfortunately. Links to it are dead but here's the github repo https://github.com/internetarchive/dweb-archive

Re:The Internet Archive and its 916B saved web pages are bac... by tonetegeatinst on Wednesday October 16, 2024 @12:03PM (#41860711)
Can confirm that issue about people only seeding what they are interested in.

I found a dataset I wanted to hoard but the authors website was gone. A dataset site had a torrent and I said great I'll just torrent and seed that and help keep the thing alive, turns out I can't find a single seeder for the torrent.

Re:The Internet Archive and its 916B saved web pages are bac... by lukas099 on Wednesday October 16, 2024 @08:51AM (#41858496)
If I help seed this DWeb and it turns out it has some copyrighted materials in it, will I be potentially held liable?
Re:The Internet Archive and its 916B saved web pages are bac... by diggan on Wednesday October 16, 2024 @10:19AM (#41859297)
You're always responsible for what you, yourself and your computer does. There is a chance EFF/some other organization could help you out in case you end up in court, but that's a maybe, not a guarantee.
Re:The Internet Archive and its 916B saved web pages are bac... by nikisweeting on Wednesday October 16, 2024 @10:35AM (#41859525)
Harder to make this argument with encrypted distributed filesystems. If I'm storing a single chunk of an encrypted blob on Filecoin, am I responsible for the entire file even if I don't know what's in it, and I'm only storing a single fragment?
Re:The Internet Archive and its 916B saved web pages are bac... by CaptArmchair on Wednesday October 16, 2024 @11:48AM (#41860482)
This depends on the jurisdiction you're in. I.e. Europe's GDPR argues that you need consent to keep someone's personal data. Encryption doesn't equate anonymization, so there's a potential liabity.
Re:The Internet Archive and its 916B saved web pages are bac... by akudha on Wednesday October 16, 2024 @01:20PM (#41861479)
This.

Until a clear, precise answer to this question is available, it is unreasonable to expect individuals to take risks and seed.

It is one thing if an organization like IA gets in trouble with the law. They have money, lawyers, name recognition and are big enough to at least fight a lawsuit, even if they lose. Who is going to help an individual if he/she gets in trouble with the law, unknowingly? Am I expected to read through tons of complex copyright law and interpret it, just so I can seed a handful of items? No thanks.

Re:The Internet Archive and its 916B saved web pages are bac... by tomrod on Wednesday October 16, 2024 @10:19AM (#41859300)
You're describing a network effects problem, specifically a collaborative game failure. Need some mechanism designers and big tech cos to jump in, stat!
Re:The Internet Archive and its 916B saved web pages are bac... by Mistletoe on Wednesday October 16, 2024 @09:30AM (#41858851)
https://www.reddit.com/r/DataHoarder/comments/h02jl4/lets_sa...

I’ve always been fascinated by this post.

Re:The Internet Archive and its 916B saved web pages are bac... by seestem on Wednesday October 16, 2024 @08:30AM (#41858315)
It is not my original idea, but it is an obvious idea to anyone who knows how the internet works, I just added it to get the discussion going.
Re:The Internet Archive and its 916B saved web pages are bac... by Cheer2171 on Wednesday October 16, 2024 @08:35AM (#41858355)
It already exists. IA has had a DWeb for 6 years. Nobody seeds it.
Re:The Internet Archive and its 916B saved web pages are bac... by seestem on Wednesday October 16, 2024 @08:43AM (#41858418)
Incentivising seeding is hard. Maybe cryptocurrencies can be useful here, but I understand not everyone likes them especially here on HN. In retrospect the ideal setup would have been if archiving was included into the core HTTP protocol.
Re:The Internet Archive and its 916B saved web pages are bac... by Cthulhu_ on Wednesday October 16, 2024 @09:26AM (#41858806)
Cryptocurrencies implies that people would pay / get paid for it... just pay the directly IA then, or your own servers. Cryptocurrencies imply someone's skimming off a lot for their own pockets.
Re:The Internet Archive and its 916B saved web pages are bac... by seestem on Wednesday October 16, 2024 @09:36AM (#41858904)
I meant more like in bitcoin how the miners get paid for mining, or how validators are rewarded in proof of stake blockchains. This techniques can be used to incentivised seeders.
Re:The Internet Archive and its 916B saved web pages are bac... by usr1106 on Wednesday October 16, 2024 @09:46AM (#41858994)
Maybe someone can invent a proof of seeding protocol? So that would bring some good to the public instead of just burning energy. Don't ask me how it would work...
Re:The Internet Archive and its 916B saved web pages are bac... by nikisweeting on Wednesday October 16, 2024 @10:36AM (#41859531)
Storj, Filecoin, etc. fill this gap but it's still really hard to earn enough to justify the effort at small scales.
Re:The Internet Archive and its 916B saved web pages are bac... by greenie_beans on Wednesday October 16, 2024 @10:06AM (#41859181)
i'm doubtful that whatever crypto incentive that is offered will make up the cost for me to DIY this in my home. which is why crypto miners scale, making crypto a centralized system. i don't care what the latest white paper says, it is still controlled by few people and not decentralized. in the same way that the ussr replicated the centralization of american capitalism, crypto replicates the centralization of trust while marketed as something else.
[dead]
Re:The Internet Archive and its 916B saved web pages are bac... by 6footgeek on Wednesday October 16, 2024 @09:04AM (#41858627)
Actually I think their wild ideas are contributing to this discussion.. I think your apparent hurt at this suggestion is a bit silly tbh.

Hear me out. I hadn't heard of the prior work you are raging about. But I am interested in decentralised tech - so, by this person mentioning their idea it got me interested to read further. Then you entered and went a bit madlad on them.. I'm not sure why when you could have just said "good question, here is everything we know about the effort to do exactly this" and then maybe that hacker would have been inspired to go read and maybe contribute. Just my 2 cents ;-)

Re:The Internet Archive and its 916B saved web pages are bac... by creer on Wednesday October 16, 2024 @01:47PM (#41861742)
If nobody seeds it (or continue development) then it's dead. Inspiration and perhaps code for the next effort sure. But not "exist" that makes a difference.
Re:The Internet Archive and its 916B saved web pages are bac... by creer on Wednesday October 16, 2024 @01:45PM (#41861723)
It seems to me the various efforts are dead or stalled. Anything in actual current development or production? IPFS was supposed to go in that direction and still exists, sure, but not to provide IA duplication (that is advertised.)