News for hackers. Stuff that matters.
Re:Obituary for Cyc by cratermoon on Tuesday April 08, 2025 @07:44PM (#43627553)
libgen is far from an archive of "most" books and publications, not even close.

The most recent numbers from libgen itself are 2.4 million non-fiction books and 80 million science journal articles. The Atlantic's database published in 2025 has 7.5 million books.[0] The publishing industry estimates that many books are published each year. As of 2010, Google counted over 129 million books[1]. At best an LLM like Llama will have have 20% of all books in its training set.

0. https://www.theatlantic.com/technology/archive/2025/03/libge...

1. https://booksearch.blogspot.com/2010/08/books-of-world-stand...

Re:Obituary for Cyc by UltraSane on Wednesday April 09, 2025 @03:20AM (#43629648)
On libgen.mx they claim to have 33,569,200 books and 84,844,242 articles
Re:Obituary for Cyc by cratermoon on Wednesday April 09, 2025 @11:03AM (#43632952)
Still an order of magnitude short of "all", and falling farther behind every year.