Yes! I really wished we would call this something more descriptive of the boring statistics involved. Artificial Intelligence is just too cool sounding name not to get excited about it. And what people don’t realize is that this is really just a statistical inference model, nothing intelligence. Machine Learning is better, but it is still not descriptive enough.
I think if we called it something like Computational Reinforcement Modelling or Iterative Weight Inference people might stop having the idea that there are machines making “smart” choices involved and finally seeing that really we are just inferring based some computations on provided data.
Really, “Machine Learning” deserves no more hype than the boring sounding Kalman Filter.
Scientists are in the best position to influence the public and push for change. It is astonishing that scientific organizations have been timid about nuclear energy, or outright against it, because of a few people with the most extreme, empirically unjustified stance.
Part of my point is that you might think that the game is "scientists vs. politicians, ignoramuses, and fear-mongers," but that's not the case. Sometimes they're all on the same side. Imagine a 4 player split-screen Star Fox, where the scientist leaves their controller and helps the others shoot down their plane because they don't think they can win. If that sounds baffling, it is.
We could have had your safe nuclear power involved in every road accident. Yeah!
Maybe a better analogy is driving a car, that is, nuclear power and self-driving cars are both vaguely like driving a car.
However, my intuition is that nuclear power has less variables than driving a car. I say intuition because I don't know much about neither.
I think the point is that self-driving cars trained on poor data are no better than poorly trained superhumans at driving.
During the cold war they launched fission reactors into orbit. That probably had more variables. Then again, since the nuclear airplane project was unsuccessful, maybe that had the most variables of all.
Putting a nuclear reactor in space (as we still do, I think the most recent Mars rover had one) greatly reduces the risk of it directly causing deaths, since it's really far away from humans for most of its service life.
If you look at the Uber accident, killing 1 person severly harmed the company.
The only extremely harmful scenario that I can imagine in self driving is a bad remote mass software update (which is sadly possible even in an otherwise great company) causing lots of accidents at the same time in different vehicles.
The OP is selling something, and it's in their best interest to spread FUD to sell it.
It's scary that the only way we know how to build something that detects pedestrians in an image with any kind of reliability is to use training data.
You: All training data has labeling issues. Me: Training data is the only way we know how to build some aspects of systems. Other people here: Some of these systems are safety-critical.
I doesn't feel like FUD to be concerned that we need to have strategies for mitigating quality issues in training data. That if you don't have anything like that, then you cannot be in the market of building a safety-critical system. As you have identified, labeling issues are always there. Kind of how human error is always there. Okay, so now that you believe that, you have to find ways to mitigate it.
But when I hang around people who do self-driving cars, they don't have a good answer to how to mitigate this risk. Some of them don't even really believe in it (which is different than being ignorant of the problem), because they argue with enough data things will get good enough. It's all really sloppy.
Instead of just saying "This is FUD", why not tell us why you feel so reassured that this is not a problem? So far, you said "this always has been a problem and always will be a problem", "the world hasn't ended yet" and "Google translate". What?
re: The world hasn't ended yet: okay, but we also haven't been building self-driving cars
The person that originally posted this thread that works for the company that's selling this FUD. At the bottom of the blog post:
"Roboflow accelerates your computer vision workflow through automated annotation quality assurance"
This is a non-issue exasperated by a for-profit corporation that's creating an "issue" they conveniently have a service to help you fix.
> tell us why you feel so reassured that this is not a problem
Because to me you're coming across as the old-school "you don't need QA if your code is perfect" type.
But for safety critical systems it's six nines reliability or GTFO.
It's not so much that "nobody gets killed by Google translate messing up" as "when people get killed by Google Translate messing up, I can't tell".
Think of the trope about idiots driving into lakes or off the road or whatever because they were following navigation. And now think about unconditionally trusting Google Maps or anything else, for say, a year. You know it's not up to that standard.
I think you can debate whether it needs to be an order of magnitude better, or multiple orders of magnitude better.
I mean, the exact same argument could be made about people. Just because you belong to Homo Sapiens doesn't imply you're an intelligent being at all times - e.g. drink a bottle of vodka and the sapiens part is gone.
That's the bar an AI system has to meet today. And one big hurdle is that, viewed as minds, AI systems are completely unlike our own. They're extremely unpredictable. On top of that, they lack self-preservation instinct (this may be indirectly a side effect of how we structure companies - the level of effort put into an AI system is a reflection of how much people working on it care about getting it right vs. getting it to market).
> they lack self-preservation instinct
I suspect that a self-preservation instinct, even in relatively dumb animals, is enormously complex. It might seem simple to us because it's so evolved, so hardwired. If I had to bet, I'd bet that a mouse's self-preservation instinct is more complex than the first-to-market Level 5 self-driving car will be.
And that's good, because I definitely don't want automated vehicles to have more than a hint of self-preservation wiring.
But the broader reason I brought up self-preservation is that in humans, it's not just about not getting killed in a crash. It's also about not having your life ruined by causing it, even if you walk away physically unscathed. A person has personal, deeply visceral stake in driving safely. A self-driving car AI currently doesn't, because it's not a mind; what it has is the reflection of priorities in the company that made it. And as it turns out, companies can kill people and get away with it, so I don't feel confident about how safety is prioritized (see also: this article).
I think it might even be more like Platonic shadows on a firelit cave wall of the priorities of the company that made it.
At the current level that self-driving is at, the cars' control software is only able to perceive and respond to situations that its programmers anticipated and specifically coded for. That could be heuristics and rules in the hard-coded bits, but, also, even the most sophisticated deep neural net out there isn't even particularly capable of perceiving, let alone understanding, things that weren't coded into its training set. Meaning that, even if the company's intention was that the car should not plow into a pedestrian holding an enormous cellophane-wrapped Edible Arrangements™ fruit basket, that intention doesn't have any bearing on the vehicle's behavior if the software instead classified that object as a plastic bag blowing in the wind.
E.g., it might be in the long-term interest of the company to not run over pedestrians. If nothing else, the PR cost is very high. But when an exec wants to be first in the field to demonstrate personal success, then hitting a made-up date becomes the priority. Which implicitly puts "not kill people" lower. Middle managers don't want to get blamed, so they'll favor a more complex, muddled organization structure. Per Conway's law, that means muddled code. So instead of the car's software reflecting "don't run over pedestrians" as a key goal, its true priorities are things like "gives a good demo", "was delivered 'on time'", and "reflects an architecture clever enough to get that architect promoted that quarter".
It's not impossible that good software comes out of this system, but the deck is certainly stacked against it.
First, the AI hype train. People think that calling something "Artificial Intelligence" implies that it is artificial, yes, but also, critically, that it is intelligent. Many enthusiastic people, and also many policymakers, don't fully realize the extent to which machine learning is constrained by both the quality and nature of its training data, and the capabilities of the larger, non-intelligent, software framework that it's being plugged into.
Second, that we have one case study - the post-mortem analysis of the fatal pedestrian collision in Arizona - that strongly indicates that commercial products are not free of the sorts problems being highlighted here, and that, unlike what others have suggested, misclassification problems aren't necessarily an issue that's isolated to individual frames and that will come out in the wash when the software is dealing with a stream of frames.
Me, I think that self-driving cars are probably a lot like nuclear power. In theory, yes, it is a great idea. In practice, there are a lot of little details that one must get right, and there seem to be a whole lot of opportunities for flaky engineering decisions and incompetent public policy, both enabled by insufficiently-tempered optimism, to scuttle the whole thing.