Re:Some AI Systems May Be Impossible to Compute
by YeGoblynQueenne on Thursday March 31, 2022 @06:23PM
(#30872891)
Just to be clear, a neural network is not "an AI". "AI" is the name of the research field. We don't have "AIs" as in Science Fiction yet, and neural networks don't do anything "just like a human". When people in AI research talk about "attention" in neural networks, that's just an anthropomorphic, and quite unfortunate, name for a specific technique in training neural nets. It doesn't mean that a machine vision system has the ability to focus its attention in the same way that humans do.
That out of the way, there are AI approaches that can explain their actions just fine without going dumb. For example, I posted this comment earlier:
https://news.ycombinator.com/item?id=30872400
about an AI system called Nook that recently won a tournament against 8 human champions of the card game Bridge. In Bridge, players must be able to explain their moves, so an AI player without the ability to explain its decisions can't play a full game of Bridge.
But when an AI gets sufficiently complex of course there won’t be explanations that make sense for those kinds of errors, because just like a human the AI is integrating lots of different bits of information that it has learned are important and it has limited capacity and sometimes it just gets its attention focused on the wrong thing and it just didn’t see the guy, okay?
Demanding that AI be explainable is fundamentally demanding that it not be intelligent.