Re:The black box is a state of mind
by YeGoblynQueenne on Monday February 05, 2018 @02:53PM
(#16311501)
What we mean when we say that (statistical) machine learning models are "black
boxes" is that they are just vectors of numbers, that have no meaning that a
human can readily understand.
Note that scale doesn't have anything to do with this. Say I show you a model
my algorithm has learned, that's basically a vector of five numbers: [n1, n2,
..., n5]. Just by looking at those numbers, you can't say anything about this
model, with any certainty: what data it was trained on, what algorithm, what
classes it was trained to discriminate or if it is a regressor- or anything at
all, really. It's just a bunch of numbers.
We can compare this to the incomprehensibility of a large software, or
engineering project. There's probably no one person who can understand
everything about an airliner- but there are many people who understand its
constituent parts, down to the level of nuts and bolts. This is not so with
(statistical) machine learning models- they are numbers without recognisable
context all the way down (although there are exceptions, this is a pretty
solid rule).
And that's why people are afraid that, with (statistical) machine learning
we're riding a machine that we don't understand and therefore do not control.
Note that scale doesn't have anything to do with this. Say I show you a model my algorithm has learned, that's basically a vector of five numbers: [n1, n2, ..., n5]. Just by looking at those numbers, you can't say anything about this model, with any certainty: what data it was trained on, what algorithm, what classes it was trained to discriminate or if it is a regressor- or anything at all, really. It's just a bunch of numbers.
We can compare this to the incomprehensibility of a large software, or engineering project. There's probably no one person who can understand everything about an airliner- but there are many people who understand its constituent parts, down to the level of nuts and bolts. This is not so with (statistical) machine learning models- they are numbers without recognisable context all the way down (although there are exceptions, this is a pretty solid rule).
And that's why people are afraid that, with (statistical) machine learning we're riding a machine that we don't understand and therefore do not control.