News for hackers. Stuff that matters.

A comment on Libgen-text: complete Library Genesis, re-packaged as text only (1.2TB). Parent comment

Re:Libgen-text: complete Library Genesis, re-packaged as tex... by VHRanger on Sunday August 21, 2022 @05:09PM (#32544471)
Yes, there's a lot better compression available these days, but not everyone has the software installed to unpack them
[ Parent | on HN ]
Re:Libgen-text: complete Library Genesis, re-packaged as tex... by unicornporn on Sunday August 21, 2022 @05:33PM (#32544681)
> not everyone has the software installed to unpack them

People downloading 2 TB of compressed books in plain text format are not exactly the definition of "everyone".

> Yes, there's a lot better compression available these days

Tell me more!

[ Parent | on HN ]
Re:Libgen-text: complete Library Genesis, re-packaged as tex... by VHRanger on Tuesday August 23, 2022 @06:50PM (#32572616)
zstd: https://github.com/facebook/zstd

Or any of the ones they benchmark against in their .README (lz4, snappy, zlib, etc.)

[ Parent | on HN ]
Re:Libgen-text: complete Library Genesis, re-packaged as tex... by yyyk on Sunday August 21, 2022 @05:15PM (#32544528)
We practically need libgen-crystal to use this anyway, so just link zstd to libgen-crystal and now it's available.

* Also, this consideration did not prevent trying .tar.xz. I don't imagine .tar.xz is that much more available than .zstd.

[ Parent | on HN ]