(To be clear by copying I mean repeating the text in your derived work, perhaps I should have been more explicit. The crux of this case is whether “copying into training set” counts as copying the whole work. )
https://en.m.wikipedia.org/wiki/Williams_%26_Wilkins_Co._v._...
A teacher copying a New York Times article verbatim and distributing it to a classroom for the purpose of analysis or criticism is likely to be held by the courts as fair use. On the other hand a teacher photocopying an entire textbook (or distributing a PDF) would probably not be considered fair use.
Both are copying educational material in full, but they have a different impact on the market. Distributing copies of an old news article to 30 students realistically isn't a direct substitute for 30 subscriptions to The New York Times. But a textbook publisher's business model specifically revolves around selling its textbooks to students - the teacher is distributing their copyrighted material to their target market and removing any incentive for the students to pay the publisher.
It makes you wonder why "Open"AI started getting into legal trouble only when they closed everything down and became a commercial project. When they were a research project, stepping on copyrights was a lot more acceptible (both socially and legally).
Plenty of works quote and cite other copywritten works and it's considered transformative because usually the work is building upon or commenting on the quote.
This is actually a huge area of debate on Youtube as Google often errs on your side of the argument (they can choose to demonetize whatever they want), even if they work is technically not infringing.