Re:OK, I can partly explain the LLM chess weirdness now
by _heimdall on Friday November 22, 2024 @09:09AM
(#42213939)
> The inputs and outputs of the model are human language, so at least there we know the system as a black box can be characterized, if not understood.
This is actually where the AI safety debates tend to lose. From where I sit we can't characterize the black box itself, we can only characterize the outputs themselves.
More specifically, we can decide what we think the quality of the output for the given input and we can attempt to infer what might have happened in between. We really have no idea what happened in between, and though many of the "doomers" raise concerns that seem far fetched, we have absolutely no way of understanding whether they are completely off base or raising concerns of a system that just hasn't shown problems in the input/output pairs yet.
What is difficult is finding some intermediate pattern in between there which we can label with an abstraction that is compatible with human understanding. It may not exist. For example, it may be more like how our brain works to produce language than it is like a logical rule based system. We occasionally say the wrong word, skip a word, spell things wrong...violate the rules of grammar.
The inputs and outputs of the model are human language, so at least there we know the system as a black box can be characterized, if not understood.