News for hackers. Stuff that matters.
Re:Show HN: Rarbg on IPFS by mmastrac on Sunday June 04, 2023 @11:28AM (#36186368)
I was curious how this works and then I saw the sqlite requests in the network tab. It's amazing to see what we have access to these days -- SQLite over HTTP over IPFS to provide a giant, censorship-resistant database!
Re:Show HN: Rarbg on IPFS by TekMol on Sunday June 04, 2023 @11:44AM (#36186506)
It's censorship resistant if enough nodes mirror the sqlite file I guess.

Is there any incentive for nodes to do so?

Is it possible to see a statistic about how many nodes mirror it?

Re:Show HN: Rarbg on IPFS by lyu07282 on Sunday June 04, 2023 @12:54PM (#36187171)
torrents work fine without having a monetary incentive, this misconception of the blockchain crowd really has to die its killing real decentral solutions
Re:Show HN: Rarbg on IPFS by bakugo on Sunday June 04, 2023 @01:02PM (#36187258)
Public torrents die all the time.
Re:Show HN: Rarbg on IPFS by TekMol on Sunday June 04, 2023 @01:44PM (#36187652)
I said "incentive". You made it "monetary incentive".

What do you mean with "torrents work fine"? Do you have any statistics on the lifetime of torrents?

Everything ever written to the Bitcoin blockchain is still available and massively distributed. It seems very unlikely anything will ever get lost during our lifetime. So I would say that is an example that monetary incentives do work?

Re:Show HN: Rarbg on IPFS by Barrin92 on Sunday June 04, 2023 @03:09PM (#36188510)
>Everything ever written to the Bitcoin blockchain is still available

This isn't useful for 99% of stuff and pushing every video and music file on earth through a global state machine with the performance characteristics of an Atari from the 80s would render the thing inoperable. The entire Bitcoin network has a bandwidth of like 1mb every 10 minutes, the total size of the blockchain is half a TB in total, people torrent more porn in the time it took me to type this.

Maintaining a global, complete history of transactions only ever made sense for one problem, double spending, and is utterly useless for sharing files.

Re:Show HN: Rarbg on IPFS by sebmellen on Sunday June 04, 2023 @06:04PM (#36190032)
IPFS is not a blockchain in the traditional sense. You might be conflating it with Filecoin.
Re:Show HN: Rarbg on IPFS by abwizz on Monday June 05, 2023 @03:22AM (#36193459)
ok.

but nobody said otherwise

Re:Show HN: Rarbg on IPFS by xtracto on Monday June 05, 2023 @12:21AM (#36192225)
They dont. Do a search of something older than 5 years in btdigg and you will most likely get only dead magnets.
Re:Show HN: Rarbg on IPFS by xnyanta on Monday June 05, 2023 @01:02AM (#36192509)
They do on private trackers where there is an incentive usually tied to ratio or seed time. It's not a monetary incentive and it works.
Re:Show HN: Rarbg on IPFS by xtracto on Monday June 05, 2023 @11:30AM (#36197688)
Aaah private trackers! I remember once I pondered to enter one of those "special groups" but bailed when I read that you had to study for some sort of interview that was required to try to join .

Lol no... I dont like interviewing for a new job, why would i agree to get tested for something as stupid as piracy? (Unless... mention me ONE private tracker that doesn't have piracy).

So for 99% of people, my point stands.

Re:Show HN: Rarbg on IPFS by nephanth on Wednesday June 07, 2023 @08:30PM (#36235581)
I remember what.cd had interviews like that. Tried and failed as a kid, that was a pain

There are definitely private trackers that don't have anything like that though (iirc they're called "semi-private trackers"). One i remember using is t411 when it was still running

Re:Show HN: Rarbg on IPFS by WeylandYutani on Monday June 05, 2023 @06:25AM (#36194510)
Sharing is caring!
Re:Show HN: Rarbg on IPFS by qersist3nce on Sunday June 04, 2023 @11:51AM (#36186557)
Can you explain bit more? Isn't the dump of last RARBG magnet links on the order of MBs? We can just download it and grep in plain text. I don't get what is the role of SQLite or IPFS here.
Re:Show HN: Rarbg on IPFS by phiresky on Sunday June 04, 2023 @12:44PM (#36187078)
This blog post explains how it works: https://phiresky.github.io/blog/2021/hosting-sqlite-database...

Disclaimer: I wrote that article and was somewhat involved in the other sqlite over ipfs project this is forked from.

Yes, for MB size files just downloading the whole thing will be faster and much easier - even if running in the browser. I'd say the boundary is somewhere around 10MB compressed (~20-200MB uncompressed). Looks like the sqlite dump used here is ~400MB in size, ~180MB compressed. Loading that in the browser would probably work but it wouldn't be too great.

Re:Show HN: Rarbg on IPFS by romnon on Sunday June 04, 2023 @01:08PM (#36187306)
this, anyone above 20 can operate.
Re:Show HN: Rarbg on IPFS by runeks on Monday June 05, 2023 @02:32AM (#36193116)
> Isn't the dump of last RARBG magnet links on the order of MBs?

As I understand it, the purpose of using SQLite is indexing and querying, ie. being able to efficiently search through the data via a website.

Re:Show HN: Rarbg on IPFS by remram on Sunday June 04, 2023 @01:10PM (#36187329)
IPFS is distributed but not anonymized, so you can send abuse requests to people's ISPs to take content down (like with torrents).

The HTTP part is also not censorship-resistant. ipfs.io has a DMCA process, and they could also be asked to reveal the IP of users.

Re:Show HN: Rarbg on IPFS by activiation on Sunday June 04, 2023 @04:03PM (#36189045)
Use Tor?
Re:Show HN: Rarbg on IPFS by tylersmith on Sunday June 04, 2023 @06:41PM (#36190242)
IPFS can be accessed through Tor immediately through a gateway, and with a little work through the ipfs client directly.
Re:Show HN: Rarbg on IPFS by remram on Monday June 05, 2023 @02:51PM (#36201225)
Tor+IPFS is much more resistant yes.
Re:Show HN: Rarbg on IPFS by bscphil on Sunday June 04, 2023 @07:56PM (#36190711)
> you can send abuse requests to people's ISPs

In practice it's very rare for me to see a direct ipfs protocol link, almost all the traffic goes through HTTP gateways (which frequently cache the content as well). Hard to imagine that they don't become a target for "hosting" pirated content if / when IPFS becomes more than a negligible platform for piracy. (A significant amount of Library Genesis traffic is already using IPFS via these same gateways.)

As you mention, there's a DMCA process for some of the gateways, but that might not be enough to ward off attention.

Re:Show HN: Rarbg on IPFS by jazzyjackson on Sunday June 04, 2023 @01:43PM (#36187641)
it's content addressed, if an ISP was so keen it would be simple to blackhole requests for a particular file (of course, one need only to change 1 bit to get a new content hash, but you have to redistribute the new file from scratch
Re:Show HN: Rarbg on IPFS by dietr1ch on Sunday June 04, 2023 @05:51PM (#36189938)
I don't know about how IPFS is implemented, but you could use content-addressed blocks underneath every file too. This way flipping a bit means that only one underlying block changes, and that the new file would share N-1 blocks with the previous one, making redistribution only require sharing a single block.
Re:Show HN: Rarbg on IPFS by SpaghettiCthulu on Sunday June 04, 2023 @09:12PM (#36191146)
Well, wouldn't they just blackhole all the relevant blocks then?
Re:Show HN: Rarbg on IPFS by dietr1ch on Sunday June 04, 2023 @10:46PM (#36191677)
Yes. Probably there's no replacement for privacy and anonymity being baked in into the system.