Wow so this is actually just a tiny subset of what was already openly available. I'm still interested in how all of this was gathered together though - looks like Anna's Archive got the books from other large datasets, but I'm still confused about the ultimate origin of the data.

From my experience as an author of a tech book, the ebook is pirated even before the official publication date, the authors only have access to the ebooks fees days after the publication date. That’s why I’ll never write a book again.

That's sad and I can see how it must be incredibly frustrating.

Perhaps this is why so many AI books are made available for free online by their authors? For example Sutton and Barto, and Goodfellow are both available online from their authors.

Perversely I am much more likely to buy a book if the authors make it available for free. I find a hard copy more useful to study and I prefer to pick a book after sampling it (as you would in a bookshop I guess). I wouldn't go as far as getting an illegal copy to preview so those books without an easy way to review are a lot less likely to be bought (by me).

Yes I agree with you on that. However not every author can negotiate with the publisher to get the permission to offer the ebook for free on their own website (I got rejected on the idea).

I guess if you’re a famous one then you can probably get that , but now for average people?

This is really frustrating, because as an author you can’t offer your own book on your own website for free, while on some third party websites it’s already been downloaded thousands of times.

Origin of the data:

- LibGen collections gathered over decades

- Sci-Hub

- Additional fresh scholar papers and books collected by Library STC

the main ebooks sources

- bought/library borrowed books with DRM removed and shared

- scanned & OCRed books from archive.org & related projects

- non-DRM bough & shared books