I commend you. Very few people actually try feeding images into these things to show how bad they are.

For a real shock at how poor performance is, try feeding frames of video in. (Video is different because it generally doesn't have carefully stereotypically framed, exposed, in-focus content).

Edit: examples of ResNet applied to video by a colleague http://blog.piekniewski.info/2016/08/12/how-close-are-we-to-...

Very informative quote:

So where do the reports of superhuman abilities come from? Well since there are many breeds of dogs in the ImageNet, an average human (like me) will not be able to distinguish half of them (say Staffordshire bullterrier from Irish terrier or English foxhound - yes there are real categories in ImageNet, believe it or not). The network which was "trained to death" on this dataset will obviously be better at that aspect. In all practical aspects an average human (even a child) is orders of magnitude better at understanding/describing scenes than the best deep nets (as of late 2015) trained on ImageNet.

>>This is not accurate however. Other important senses in use include proprioceptive, hearing and tactile feedback from wheels. In addition to vision and the improved dynamic range of eyes, there is the important fact that human vision integrates a world model into expectations. Human vision also models time and motion which help manage where to focus attention. Humans can additionally predict other agents and other things about the world based on intuitive physics. This is why they can get on without the huge array of sensors and cars cannot. Humans make up for the lack of sensors by being able to use the poor quality data more effectively.

Yes, but all those things you listed are very imperfect and can increase the risk of mistakes. For example, someone two lanes over honking their horn can cause a momentary distraction for you ("are they honking at me?"), causing you to not notice a cyclist cutting in front of you. And so can a dancing clown on the sidewalk.

Very much so and that's why we don't want our cars to perfectly replicate humans. More sensors plus a limited and narrow AI is better than just vision and a smarter AI.

> This is true but only in a limited sense.

Isn't what we need very limited anyway? What the cars need is recognition of obstacles and the type of the obstacle in a very limited range. Basically when something is on the road, it doesn't matter whether it's a moose or a deer - you slow down and avoid, or brake depending on the environment.

"what is in this picture" classifiers don't seem like a good algorithm to use in that case. Object detection / feature extraction seems to be much closer.

Eh. Is it clouds or a truck, or a shadow on the road or a moose? A tree besides the road or a cyclist) Hard to tell without really good classifiers.

I guess that with a lidar or some kind of 3d perception, you can relax the demands on the classifier a bit and ignore really flat things and the sky...

Object detection is considered a more difficult problem than image classification in the vision community. So I'm not sure what the point is there.

> The key difference is learning in animals occurs by breaking things down in terms of modular concepts, so even when things are not recognized new things can be labeled as a composition of smaller nearby concepts. Machines cannot yet do this well at all and certainly not as flexibly.

Actually, that's pretty much what deep learning is doing. For instance: https://papers.nips.cc/paper/5027-zero-shot-learning-through...

That paper was from a few years ago, I think the state of the art is better now, but it's trying to do exactly what you're talking about. More broadly, what you're talking about falls under the umbrella of transfer learning (that is, a model's ability to learn helpful information about task Y by training on related task X, preferably by learning and sharing useful features.)

I'm talking about learning modularly. Children can categorize things they've never seen before by inventing labels on the spot, they're not limited to selecting from a preexisting set (e.g. a lion is a "big cat"). They can recognize novelty and ask questions if nothing they know quites fit.

As a human, you are able to learn the general concept of leg and understand it, even if it is in a context you have never seen before and an object you've never seen or never seen used in that way before. Everything you learn, is also as part of a set of relations. Each of individual concept modified in a precise manner as you learn something new about any of them. A big part of human intelligence, from the simple naming of things, to the highest levels of science is taking parts of things you know and putting them together in novel ways.

In Neural nets this: https://arxiv.org/abs/1511.02799 is in line of what I mean.

Well, not with traditional feedforward networks (LeNet, etc.). You can't run the classifier and find tires, then wheels, and then a car; but you do get composition of features.

> not with traditional feedforward networks (LeNet, etc.)

I'd argue they are implicitly doing this.

> You can't run the classifier and find tires, then wheels, and then a car;

Why can't you run a classifier for tires, one for wheels, one for cars, then combine their outputs for a final classifier maybe based on a decision tree? You can train all the networks at the same time and it will give you a probability distributions for all 4 outputs (tires, wheels, cars, blended). What am I missing?

That would just be your opinion. It has not been shown. It's still an open research question over what neural nets are actually learning in their intermediate layers.

You're going to need large amounts of fine-grained labeled data for each category. You've also just manually determined some sort of (brittle) object ontology. What if there are only 3 tires? What if there are four tires on the road but no car? All sorts of edge cases, and all you've done is train a classifier for cars, not actually solved driving in any meaningful way.

Doesn't scale. You don't have N brains to compose every representation.

Agreed -- If you get away from traditional feedforward networks by adding recurrence throughout, then at least there is some chance of learning scale-free features and compositionality.