Many moons ago when I worked in the finance sector, I noticed that a huge amount of work in the industry appear to comprise many groups of humans writing a elaborate stories around a few tables of numbers, while a bunch of other groups were trying to extract the numbers from the text again into some more usable tabular form again. Always seemed like a huge waste of human time and energy to me, best if it can be efficiently automated.

I've joked it's like the lecture scene in "Real Genius": https://www.youtube.com/watch?v=wB1X4o-MV6o

The problem is, there is value in: A) Generating content by bot B) Generating summaries by bot

It's just that the "lossiness" of each conversion step is going to be worrisome when it comes to the accuracy of information being transmitted. I suppose you can make the same argument when it's real humans in the chain.

However, my fear is that we get into this self-feedback loop of bot-written articles that are wrong in some non-obvious way being fed back into knowledge databases for AIs, which in turn are used to generate articles about the given topic, which in turn are used in summaries, etc.

I think traditionally referring back to primary sources was a way of avoiding this game of telephone, but I worry that even "primary sources" are going to start being AI-cowritten by default.

Speaking of primary sources, if you ask the chatbot to reference some facts, it might very well make up plausable sounding sources. Maybe the reference doesn't exist at all. Maybe the reference exists but its by a different author. Maybe the reference exists and its the correct author, but the quote isn't found in the book at all, and to verify now you need to get ahold of the book in some form. It just seems like a chore, all to end up not entirely confident that what you have is true signal anyhow.