>But that's not what's happening. You can't "crack open" a model to find the bitmap of a piece of training data. It's not there any more than a painting you've seen is "in" your brain. A model sometimes creates things that look similar to existing pieces if it got a ton of copies of the same image, for the same reason that most people's artistic rendition of a tree is going to be more accurate than their rendition of an anteater.

In other words, a technology that has never existed before and doesn't follow any of the categories and schemas we dreamt up to organize the world. So we need to make new rules.