Re:The Computer Scientist Peering Inside AI’s Black Boxes
by janalsncm on Monday May 01, 2023 @11:21PM
(#35782920)
It would be very satisfying if we could have an explanation like that. But that’s far from guaranteed, unless that was part of the explicit design of an MMOE for example. Very likely if you ask “what part of the network recognizes faces” the best answer is “the whole thing”.
Re:The Computer Scientist Peering Inside AI’s Black Boxes
by int_19h on Saturday May 06, 2023 @03:20AM
(#35839024)
There's some evidence for emergent "modules" inside NNs.
https://www.alignmentforum.org/posts/j84JhErNezMxyK4dH/llm-m...
But “this portion of the network appears to recognize faces” is still a useful abstraction. In the same way it is for neurology. Even if both NNs and brains sometimes see faces that aren’t there because the network isn’t “recognizing faces” but a system which signals on a complex set of heuristics that mostly detects faces.
How does the answer change if I only want 99% confidence and accept the existence of illusions? — can we describe the bulk of the behavior in a summary?