> Ehh, in rare cases in can though. If you have someone sign an NDA, they can't go and publish technical details about something confidential that they were trained on. For example, this is fairly common in the tech industry when we send engineers to train on proprietary hardware or software.

And I think nearly everyone would agree that it would be perfectly fine and reasonable for an AI trained on a proprietary corpus of information to produce copyrightable/secret material in response to questions.

Just because I built an internal corporate search tool, doesn't mean that you get to view its output.

The question at play here is when the AI is trained on information that's in the public commons. The 'teacher' analogy is, in this sense, a very good one.