Human brains can explain themselves. What's a black box about LLMs is that they can't. They don't know how they arrived. They didn't arrive at anything. They don't have access to their internal representations. They don't think.