News for hackers. Stuff that matters.
Re:LibGen's Bloat Problem by Retr0id on Sunday August 21, 2022 @09:51AM (#32540186)
In an ideal world, every book could be given an "importance" score, for some arbitrary value of importance. For example, how often it is cited. This could be customised on a per-user basis, depending on which subjects and time periods you're interested in.

Then you can specify your disk size, and solve the knapsack problem to figure the optimal subset of files that you should store.

Edit: Curious to see this being downvoted. Is it really that bad of an idea? Or just off-topic?

Re:LibGen's Bloat Problem by DiggyJohnson on Sunday August 21, 2022 @10:40AM (#32540645)
Not to sound blunt, but answering your question on the downvotes (which you probably didn’t deserve, especially without reply).

The concept of an importance score feels very centralized and against the federated / free nature of the site. Towards what end?

If the “importance score” impacts curation, I am strongly against it. Not only is it icky, but how is it different than a function of popularity?

Re:LibGen's Bloat Problem by Retr0id on Sunday August 21, 2022 @10:49AM (#32540731)
I'm not suggesting reducing the size of the LibGen collection, I'm thinking along the lines of "I have 2TB of disk space spare, and I want to fill it with as much culturally-relevant information as possible".

If the entire collection were availble as a torrent (maybe it already is?), I could select which files I wish to download, and then seed.

Those who have 52TB to spare would of course aim to store everything, but most people don't.

Just as the proposal in the OP would result in the remaining 32.59 TB of data being less well replicated, my approach has the problem that less "popular" files would be poorly replicated, but you could solve that by also selecting some files at random. (e.g. 1.5TB chosen algorithmically, 0.5TB chosen at random).

Re:LibGen's Bloat Problem by liberalgeneral on Sunday August 21, 2022 @11:12AM (#32540952)
I don't think you've deserved the downvotes, and I don't think it's a bad idea either; indeed some coordination as to how to seed the collection is really needed.

For instance phillm.net maintains a dynamically updated list of LibGen and Sci-Hub torrents with less than 3 seeders so that people can pick some at random and start seeding: https://phillm.net/libgen-seeds-needed.php

Re:LibGen's Bloat Problem by ssivark on Sunday August 21, 2022 @11:17AM (#32541010)
Seems like a perfectly good idea to me! Basically how proposing that we decide caching by some score, and the details of the score function be tweaked to handle the different aspects we care for.

I wonder whether this idea is already used for locating data in distributed systems — from clusters all the way to something like IPFS.