ChatGPT's opinion on the matter is completely worthless, unless it was also trained on an accurate description of its training process (it wasn't). Language models do not even have access to their own "thought process" - if you ask it "why" it said something, you will get a post-hoc rationalization 100 percent of the time because the next-word prediction only has access to the same text that you see. The rationalization might be incidentally correct, or it might not - either way it contributes no real information about the model's internal state.

There's an interesting theory that this is all that consciousness is: one part of the brain trying to explain the decisions of another part, a part into which it has no special insight.

That interesting theory is called the bicameral mind, and as far as I know it's widely considered pseudoscience and not taken seriously in any scientific field.

Also, it doesn't describe at all the way LLMs work, so it isn't even applicable.

So just like humans

No, not like humans. Humans have access to their own thought processes, and are capable of introspection.

Why do people write these kinds of "answers" that the model gives. It's not like the model knows why it's doing anything.

Not understanding how LLMs work.

Ah such condescension! I was careful not to provide an opinion, but replicate the answer as is. The intent was not to treat that answer as fact; but I thought that response was pretty revealing and in fact supported the parent comment on LLMs having been trained on copyrighted materials. The response chatgpt provided was that OpenAI models are designed to create original content "without directly referencing copyrighted characters". If it was creating original content it needn't have referred to the constraint with respect to avoiding directly referencing the copyrighted characters.