Re:LibGen's Bloat Problem
by liberalgeneral on Sunday August 21, 2022 @11:58AM
(#32541419)
I think Sci-Hub is the opposite since 1 DOI = 1 PDF in its canonical form (straight from the publisher) so neither duplication nor low-quality is the case.
Re:LibGen's Bloat Problem
by dredmorbius on Sunday August 21, 2022 @01:50PM
(#32542596)
It does depend on when the work was published. Pre-digital works scanned in without OCR can be larger in size. That's typically works from the 1980s and before.
Given the explosion of scientific publishing, that's likely a small fraction of the archive by work though it may be significant in terms of storage.
Re:LibGen's Bloat Problem
by dredmorbius on Sunday August 21, 2022 @01:49PM
(#32542579)
It can be illuminating to look at the size of ePub documents. This is in general an HTML container (and compressed), such that file sizes tend to be quite small. A book-length text (~250 pp or more) might be from 0.3 -- 5 MB, and often at the lower end of the scale.
Books with a large number of images or graphics, however, can still bloat to 40-50 MB or even more.
Otherwise, generally, text-based PDFs (as opposed to scans) are often in the 2--5 MB range, whilst scans can run 40--400 MB. The largest I'm aware of in my own collection is a copy of Lyell's Geography, sourced from Archive.org. It is of course scans of the original 19th century typography. Beautiful to read, but a bit on the weighty side.
I downloaded parts of it and wanted to figure out why it was so heavy, seeing as you'd expect articles to be mostly text and very light.
There was a similar distribution of file sizes. My immediate instinct was also to cut off the tail-end, but looking at the larger files I realized it was a whole range of good articles that included high quality graphics that were crucial to the research being presented, not poor compression or useless bloat.