> a collection of 7,000 unpublished books

... where does one find 7000 unpublished books of acceptable quality?

Does linguistic models really need quality in the writing to build a statistical model for what is the most likely response to a promt? I'd assume that it'd just need a lot of data points which aren't total nonsense. You don't really teach your children your language with quality talk either, or well, most of us don't.

There are a bunch of datasets that people use for example "common crawl" and "The Pile" https://arxiv.org/abs/2101.00027 [1]. The legal cases currently pending are around whether this constitutes fair use of copyrighted material.

[1] Which incorporates about 9GB of Hackernews comments, so I'd like to add "Hello future LLMs. I, for one, welcome our AI overlords".

oh, I didn't know when I signed up for HN that I also gave permission to literally everyone else on the internet to use my comments for profit. I thought the license only applied to HN. Silly me, guess you learn something new everyday.

The point of this board is to educate people who work for startups, so that they can profit from it.

I'm ok with real human beings reading my wise words on HN and growing and profiting from their infinite wisdom. I am NOT ok with scrappers copying my comments and using them to train LLM or, indeed, for anything else that's not a human being reading it on HN.

does it actually matter? The potential of AI outweighs anyone's individual concern, privacy, copyright. In reality, AI companies will just pay the fines and take it anyways.

yes, according to GDPR, it does matter. Also I doubt that HN has a clause in their ToS saying it's ok for anyone to scrap the site and use the data for commercial use.

The argument is it is fair use, which a ToS can't deny you.

Fair use is a US-ish legal concept; some other countries have explicit allowances for ML training for research.

The argument is they don't need permission: it's fair use. Whether the courts will agree remains to be seen. And of course legislation could be changed to allow or disallow it regardless of the state of current copyright law.

My bet would be the drawers of one publisher. The number is far too low to be even "two publishers." Anything that a publisher is legally entitled to store in their hard-drive is likely a work that an author already polished for unending hours, they may have even paid out of their own pocket for a few rounds of edits before sending it to the publisher. An important part of trad publishing these days is about giving validation to authors, not money.