Re:An understanding of AI’s limitations is starting to sink in
by a1369209993 on Monday June 15, 2020 @10:53PM
(#23535499)
> If you only train it on examples of 0-9 then those are the only outputs it's going to give.
Exactly.
> 11 outputs: 0-9 and "non-digit"
IIRC, this doesn't work (or not well) because the net tries to find similarities between the various members of the set "everything except these ten specific things", but you could just require low confidence for all digits on non-digit inputs as part of the gradient descent function.
The problem is more that - if people with decision-making authority trust the AI to not be insane and evil by default - failure modes like this have to occur to you before the AI starts misbehaving in production.
If you only train it on examples of 0-9 then those are the only outputs it's going to give. If you fed a "P" into such a net then the outputs would be the degree of similarity of that "P" to each of the (0-9) digits it was trained on. You could of course threshold the output and ignore any prediction with confidence less than, e.g., 90%.
If you wanted the net to do a better job of rejecting non-digits, or at least some specific ones, then you could include a bunch of non-digit examples in your training data (so now your net has 11 outputs: 0-9 and "non-digit"), then hopefully - but not necessarily - it's highest confidence prediction will be "non-digit" when presented with a non-digit input.