This isn't about the output for content generators or about the abstract numeric weights that they operate over. That's more complex and a largely open question.
But this is literally about indiscriminately distributing copyrighted works in a large, convenient archive while arguing that it's okay because you normalized the formatting a bit and because you suspect that some people might find "fair use" value in it.
Meanwhile the complainant (such as GRRM) forgets that often passages from said articles and books are strewn throughout the Internet. Of course chat GPT can drop passages from the GoT books; there's several entire fucking wikis for that franchise that reference passages, quotes, details etc.
Same goes for news articles, passages of which are often quoted by other sources or websites.
Not that chatGPT has reproduced many works in whole, but it's an interesting logic problem for fair use law: if I have copyrighted article X, but websites ABCDEF all quote various passages of my article (ie fair use, critique, etc) and then ABCDEF is used to train an LLM, if the LLM can _reassemble_ the article from quoted passages without referencing article X itself, is it copyright infringement or fair use?
So in that sense I understand the response that "they don't violate copyright" by studying the material. Again, I don't pretend to be a lawyer, and not every law has to follow my logic.