> Besides, if LLMs only recycled training data with no changes, they'd just be really bad search engines.
I will admit that in my experience, LLMs tend to be really good at tip-of-my-tongue type stuff. There are certain very particular types of queries that LLMs seem to greatly excel at, and they're mostly where the words I want aren't in the words that I am using. I can just spam vibes into the prompt and have an LLM give me words/phrases that exactly fit what I am looking for, even if I couldn't recall any of the proper terms that would allow good results to turn up from a search engine.
Besides, if LLMs only recycled training data with no changes, they'd just be really bad search engines. Generative AI was created initially to improve training, not for human consumption - the fact that it did improve training shows that the result is greater than the sum of its parts. And since nowadays they're good enough to pass for conversation, you can even observe that on your own by asking a question that doesn't appear anywhere on the training dataset - if there's enough coverage on that topic otherwise, I've seen them give very reasonable answers.