Aren't you ignoring the human failure modes here? If self-driving cars were safer than human-driven ones, we would definitely be in the positive, and even more so for "safer than flying". There would simply be no downside (in terms of failure mode/rate) to begin with.
I get your overall argument, but for this point you are ignoring precisely what you are advocating for - you are looking at the solution in isolation, not comparing to the alternatives.
You'd have to convince people they were terrible drivers and that driving was super dangerous to get most people to put their kid in a self driving taxi in traffic. I'd agree there is a cultural change effect in play, as cars were farcically unsafe (and yet still 1000x safer than horses) for the first 60 years of their use, but the business model where people switch to self-driving is optimistic.
That isolated individual decision making will be the thing that scales. It's a question of perception of risk/reward, which is very different from statistical risk modelling.
The AI/ML will be great, the products? Whole different class of problem.
I don't think so. All things equal, who wouldn't want to spend their time doing something more fun than navigating traffic? Whether it can be affordable to everyone is a different question, but you wouldn't have to insult anyone's driving skills to offer incentives for a switch.
Also, "self-flying cars" have about 100x more problems that affect adoption (aside from having virtually nothing in common with self-driving ones). Having to constantly generate 10 m/s² of upwards acceleration is really expensive. You can't go to work in a flying car. People are huge wimps about crashing. Basic stuff like this. I mean, I can't/won't stop you leaving the self-driving car business for the self-flying car business, but it seems incredibly ill-advised to me.
This is very insightful. I have a friend working on making audio searchable by transcription using ML. I immediately thought that was a good idea unlike many other ML ideas, and "asymmetric upside" is a good summary why.
The metric is not the ROC curve, the metric is the overall value the algorithm generates in aggregate, based on the unique business situation as a function of risk.
When you look at a trading bot, it is not evaluated on its predictions of market moves, but on how much money it makes - a higher order effect of using the method over time vs. others.
You have to look at the symmetry (or distribution) of risk in the problem it's being applied to. (famously described in a ROC vs. Indifference curves blog post)
In the case of cancer detection, ML is immensely valuable in alerting people who might not otherwise have been tested, but terrible as a diagnostic tool when it is substituted for human judgment. Not because it is inaccurate, but because the consequences of it being wrong are fatal to the patient, where the benefits of it being right are life saving. You need to have a problem with asymmetric upside to benefit from ML. That is, the downside to using it can't be serious, because the machine cannot hold risk or accountability, and no sane organization would expose themselves to catastrophic liability for a probabilistic algorithm being wrong.
This view pisses off managers, executives and investors because most of the ML products out there hide their downside. Luckily, most customers are smart enough to recognize this - which is why AI companies are having such trouble finding PMF.
Self driving cars are another example of a problem with a catastrophic failure mode. You could argue that if we made self driving cars as safe as flying, that would be acceptable. Except the net benefit/upside of a self driving car is not sufficient to justify the downside of its failure mode.
Self-flying vehicles will be more acceptable much sooner than cars because most people don't fly. The upside benefit anchors people to a higher degree of risk appetite than for something they already do. If I were in the self driving car business, I'd switch to the self flying car business for that reason.
From a management perspective, AI/ML does not produce something you can manage. It's not like there are rational levers and incentives you can adjust to achieve outcomes. It's a curve, you tweak weights and try to keep it producing aggregate net benefit. If you are not directly engaged in that process, it's a lottery with outcomes distributed along its a ROC curve. In many ways, it is a substitute for management.
AI/ML is useful for marginal fraud detection, policing of various kinds, and other relationships where you are in a position of power over a distribution of outcomes. It is not something you can be honestly served by.
This is an unpopular view because it bursts the AI/ML bubble, where it has been an excellent source of greater fools for investment, but if you have money in the space, the question to ask is, does the problem this company is solving provide a net benefit that compensates the subjects to this system for its failure mode?
Not how low can you get the MTBF FP/FN rate, that's the hustle. The real question is whether a given individual user or subject of this system can afford the cost of it being wrong. If you don't have that answer, metrics are useless.