Interestingly in the article, they try to differentiate the system from machine learning:

Advances in machine learning have been made by training a computerized mimic of a neural network on a given task. Though the networks are successful, they are also a black box because it is hard to reconstruct how they achieve their result.

“This has given neuroscience a sense of pessimism that the brain is similarly a black box,” she said. “Our paper provides a counterexample. We’re recording from neurons at the highest stage of the visual system and can see that there’s no black box. My bet is that that will be true throughout the brain.”

I believe it is the case that deep NNs at the highest/deepest levels have interpretable feature-specific cells as well. Not sure what they think the difference is.

At this stage I don't think "black box" is a very fair description. We now understand a fair bit about how artificial neural nets encode information and calculate things. And what we don't understand, we can still see and study the processes.

We cannot be entirely sure of the behavior in any situation that is sufficiently different from anything that was presented during training. You can't necessarily expect a "common sense" response in that case, and in this sense the neural networks are black boxes.

The same applies to any complex system. 'Black box' means 'not inspectable', not 'anything which doesn't have predictable behaviour over all conceivable inputs'.