News for hackers. Stuff that matters.
Re:Open Problems in Robotics by fluffy87 on Wednesday August 19, 2020 @02:53AM (#24207861)
This is the scariest part of using machine learning as an engineer on any practical application as well.

Without an error bound, ML can’t be in charge of anything that could put human lives at risk.

This is also why I don’t understand all the hype about FSD / L5 autonomous driving. We don’t even know yet if such error bounds even exist, so we don’t even know if machine learning is even the right tool for FSD yet. All certification entities for control systems that put human lives at risk in aviation, automotive, etc. require those error bounds. So it actually doesn’t really matter if Tesla comes up with a “maybe L5” system, without right error bounds, their cars won’t be certified as L5 and drivers will need to keep hands on the steering wheel.

Re:Open Problems in Robotics by jaaron on Wednesday August 19, 2020 @03:02AM (#24207904)
What do you think the error bounds are for a human?

I know it sounds like a flippant question, but for certain applications, if we can get a model that's better than human, then it doesn't need to be perfect.

And they way we currently do this in all sorts of ways is to pair a human with a computer so that they each do what they're best at. It doesn't have to be about full automation.

Re:Open Problems in Robotics by de_watcher on Wednesday August 19, 2020 @05:30AM (#24208644)
Human is like your ancient software that was here since forever and somewhat worked fine. So everyone is used to it.

(the difference from an actual software is that humans are based on some crazy nanotech from the future that nobody can completely control)

Re:Open Problems in Robotics by tomxor on Wednesday August 19, 2020 @09:32AM (#24210159)
> humans are based on some crazy nanotech from the future that nobody can completely control

Excellent summary :D

Re:Open Problems in Robotics by tomp on Thursday August 20, 2020 @02:47AM (#24220557)
This is another one of that "extreme tail risk" scenarios, like climate change and GMOs, that people have wildly different and contradictory reactions to.

Sure, the "legacy" intelligence / climate / food could also have extreme tail risks, it's just that it's been tested for 100s of millenia... whereas new technology might be better in the average case or even 99th percentile, but the 1% (or 0.0001%) is unknown and potentially much worse.

However, it seems to me that people resolve this more along ideological / political lines than with any kind of rational reasoning.

Re:Open Problems in Robotics by jnxx on Friday August 21, 2020 @12:51PM (#24236584)
> "extreme tail risk" scenarios, like climate change

Climate change isn't a "tail risk". It is a hard wall our civilization is approaching fast. If we do not solve it, it will undo the conditions we depend on to live.

Re:Open Problems in Robotics by burntoutfire on Wednesday August 19, 2020 @03:04AM (#24207913)
What about "practical error bounds", i.e. testing the system through millions/billions of miles driven?
Re:Open Problems in Robotics by imtringued on Wednesday August 19, 2020 @03:36AM (#24208074)
Empirical evidence has shown that all Tesla cars have the same failure mode when a stationary obstacle is on the highway. This is not some fringe failure that happens because of wrong classification, it's how the Tesla Autopilot was designed to function. Tesla has claimed explicitly that it's the driver's responsibility to avoid such a situation.
Re:Open Problems in Robotics by azernik on Wednesday August 19, 2020 @06:48AM (#24209009)
Tesla is not a good representative of the field; they are waaaay behind Waymo (Google) and Cruise (GM).
Re:Open Problems in Robotics by tuatoru on Wednesday August 19, 2020 @03:24AM (#24208007)
Miles driven is a useless metric. Stick your vehicle on a treadmill and have it drive a billion miles. What does that tell you?

Edit to add: a better metric would be something like "billions of decisions made where human life was at stake".

Re:Open Problems in Robotics by gspr on Wednesday August 19, 2020 @05:19AM (#24208590)
It's pretty clear the they meant "distance driven in ordinary conditions where humans normally drive cars".
Re:Open Problems in Robotics by bluGill on Wednesday August 19, 2020 @09:02AM (#24209917)
That isn't the case though. Gm and Google both tell you that their incidents per mile (I think they use 100000 miles or something) is constant, because as they get better they test in harder situations. A automatic car on a desert freeway (no traffic) is easy compared to freezing rain during the afternoon rush hour in Minneapolis. (afternoon implying most people were out and so the decision to stay home isn't available). That is just on hard situation that comes to mind, I think those doing self driving cars know of others.
Re:Open Problems in Robotics by thrwyoilarticle on Wednesday August 19, 2020 @10:42AM (#24210861)
This is how some hardware is already certified for use in safety critical systems. ISO 26262 calls it 'proven in use'.
Re:Open Problems in Robotics by redis_mlc on Wednesday August 19, 2020 @05:01AM (#24208502)
> testing the system through millions/billions of miles driven?

Makes sense doesn't it?

Well, actually no. The Google cars drive the same route every day in Mountain View with no deviation, so those millions of miles are really the same 10 miles over and over. Even seen 3 in a row behind each other.

Fools the regulators, and apparently you, though.

Re:Open Problems in Robotics by throw149102 on Wednesday August 19, 2020 @08:25PM (#24218514)
Deaths are often measured in deaths per 100,000,000 miles driven, with a current measurement of 1.18 deaths per 100 million miles[1]. If a car is driving 60 mph 24/7, it takes 190 years for it to drive 100,000,000 miles. Let's assume that, in a decade, 1% of cars are on autopilot. There are 37,000 deaths per year in the USA due to car accidents[1]. Therefore, in order for Tesla/Google/GM to claim that it is equivalent to normal cars, it has to drive 370 * (100 million/1.18) ~= 31,300,000,000 miles, with only 370 accidents. If $COMPANY really had, say, 400 accidents, then it would have to make up the additional miles. In specific, it would have to make up 30 * (100 million/1.18) ~= 2,500,000,000. That means a fleet of (190 * 25) = 4,750 cars driving 24/7 for an entire year. That's not impossible for $COMPANY to do, but it is starting to become unreasonable. What's more likely is that $COMPANY just straight up lies about its incidence rate to regulators. But that's hard to do, considering it's going to be public if someone gets hit by a car.

I don't think its possible to fool the regulators, but even if it was possible there are dramatic consequences for lying. Volkswagon tricked regulators into thinking their car didn't emit as much as it really did in 2007-2015, and ended up having to pay 33.3 billion dollars in fines and returns[2]. Volkswagon stock went from a high of around 27 to 12. 5 years later, it's only recovered to around 16. The massive lawsuit that would be formed if Tesla's (or other automated car maker) car was found to be unsafe would likely be even larger. Furthermore, it would crush the self-driving car business semi-permanently. VW at least still gets to make environmentally safe cars, Tesla and Waymo's main value proposition is dead in the water.

Basically, even though there are incentives to lie and cheat, I think the incentives to try to make something that is safe is far, far greater. And while I'm sure there are Google cars driving the same route every day, I'm sure there are also tests being done in all sorts of conditions all over the planet. I think that practically, socially, and politically speaking, we can use billions of miles driven. The stakes are way too high to lie.

[1] https://en.wikipedia.org/wiki/List_of_self-driving_car_fatal... [2] https://en.wikipedia.org/wiki/Volkswagen_emissions_scandal

Re:Open Problems in Robotics by m0zg on Wednesday August 19, 2020 @03:07AM (#24207924)
> Without an error bound, ML can’t be in charge of anything that could put human lives at risk

Humans don't have "error bounds" either, and you trust them just fine.

Re:Open Problems in Robotics by b3kart on Wednesday August 19, 2020 @05:54AM (#24208753)
Two counterpoints here:

1) Humans can estimate their own uncertainty. Ask a person to show how long a meter is, and they'll give you an estimate. Then ask them to show you the "error bounds", i.e. what they're "quite certain" the meter is longer than and shorter than. You are likely to get sensible bounds. Now, humans aren't amazing at this, but the brain does have capacity for estimating how uncertain it is.

2) No, you don't really trust humans. This very fact that humans are often imperfect in estimating their own uncertainty makes us very stupid sometimes. How many times you were sure you know something for a fact, for it to turn out to be completely false. This is why society tries to not put too much responsibility into the hands of a single person, or at least to provide help and/or safety mechanisms if that is the case.

Re:Open Problems in Robotics by m0zg on Wednesday August 19, 2020 @04:51PM (#24216256)
Ask a programmer to estimate how long it'll take them to code something to see if we really have error bars on complex functions. The margin will be so wide as to be completely useless.

We do have error bars on simple measurements already. They're right there in the data sheet for the sensor. What you're asking for are error bars on things several levels distant in the layers of abstraction. Humans suck at that. We only cope the same way machines do: through constant negative feedback.

Re:Open Problems in Robotics by snovv_crash on Thursday August 20, 2020 @09:19AM (#24222907)
Programmers can give you good error bounds. Ask what the best and worst case is for solving a problem, and 80% of the time they will be within those bounds.

The problem is that management doesn't want to hear about the worst case, because it looks bad for them politically.

Re:Open Problems in Robotics by foerbert on Wednesday August 19, 2020 @03:25AM (#24208013)
Humans are capable of generating and understanding creativity and complexity that are simply impossible for non-AGI automation. Even then, we don't just let people figure things out for themselves. We put them through training, and then test them. Even after that, we make them liable for negligence.

I don't think it's an obvious conclusion that error bounds aren't important for automation because they aren't calculable for a human. They are just very different beasts.

Re:Open Problems in Robotics by m0zg on Wednesday August 19, 2020 @04:36AM (#24208383)
My point is not that it's not important. If someone comes up with a rigorous way to obtain error bars, that's great! I'll take it! My point is that trust should not and will not depend on it. How do you even quantify something like this to a layperson in order to persuade them?

Let's do a thought experiment: let's say, we had a self-driving car that's verifiably 10x better than human on average, yet does not provide "error bars". I know we don't have one now, and unlikely to have one in the foreseeable future, but bear with me here, for the sake of argument.

Would you trust it, rather than a random human Uber driver?

FWIW, I'm amazed that driving cars manually is perceived as normal every time I drive one. I can easily accelerate 2+ metric tons of metal to 100+ MPH, and get distracted, launching this deadly projectile with me inside into oncoming traffic. Most roads have _no dividers_. This does happen from time to time, lots of people die. Nobody gives a shit.

Humans suck so bad at so many things that robots will be better than them at a lot of fairly unconstrained tasks in the next decade or two. And I'm pretty certain they won't have error bars while doing what they do. Humans don't.

Re:Open Problems in Robotics by YeGoblynQueenne on Wednesday August 19, 2020 @06:54AM (#24209044)
The other poster was talking about error bounds not error bars. That means they want to have a number that tells them how often a system fails and ideally a model of how and when it fails. So they want to have a system that comes with some guarantees. It works X% of the time, fails 100-X% of the time.

Note that X doesn't need to be high, or higher than a human, for example a self-driving car with a 60% failure rate is fine as long as we know when and how it fails. It's this "when and how" that's completely missing from self driving cars.

And in machine learning in general we just don't have any guarantees that a model trained on some dataset will perform the same way in the real world.

Re:Open Problems in Robotics by foerbert on Thursday August 20, 2020 @01:57AM (#24220339)
I think a lot of the nuance has been lost here. Even throwing around phrases like "10x better than a human" is already implying a lot of very vague measurement or knowledge. In what ways, and in what environments, is it better?

I've dipped my toes into robotics a few times, and every time I end up being reminded of just how painful it is. Even when you're working in an idealized simulator, it's extremely easy to find a little edge case that causes completely bizarre behavior. And moving out of the simulator only makes things far, far worse.

It's really easy to forget about those sorts of details and brush it away as just things to be solved while developing the automation. But they don't just go away so easily. Error bounds are there to help manage these issues and ensure we know how to best use the automation.

In many instances I think people would tend to prefer the Uber driver in a moderate reading of your scenario. A human driver is likely to perform somewhat consistently and predictably. If they drift around the corner and leave a long skid mark in front of your house you can make some assumptions about how they are going to drive. If you get in and see them struggling to keep their eyes open, you can again make some assumptions about their performance. A well-rested and safe driver is extremely unlikely to suddenly throw themselves into on-coming traffic with no warning. You can refuse or stop the ride if you judge you are not safe.

Automation is a different beast. It's liable to fail in ways that a human driver would not. It may be performing wonderfully until something a human would not even notice happens, and which point it may indeed throw you into on-going traffic. For example, look at adversarial examples in deep learning. As a passenger in this case you don't have a way to judge your own moment-to-moment safety. Even if it is safer on average, the sheer unpredictability and the resulting stress is likely going to eat significantly into any gains.

Re:Open Problems in Robotics by de_watcher on Wednesday August 19, 2020 @05:38AM (#24208684)
A human whose life is at stake controlling a car here at this moment vs a human who is far in space and time controlling someone else's car by programming it.
Re:Open Problems in Robotics by imtringued on Wednesday August 19, 2020 @03:41AM (#24208106)
Humans have a mental model of humans, they can predict what another human is going to do with reasonable accuracy. Robots are wholly artificial so there is no existing mental model we can apply to them to understand why and how they act.
Re:Open Problems in Robotics by mhb on Wednesday August 19, 2020 @09:15AM (#24210017)
You have no mental model of the crazy person who drives his car into oncoming traffic.
Re:Open Problems in Robotics by tsimionescu on Wednesday August 19, 2020 @05:24PM (#24216747)
Sure, and that's why we try not to let people with altered states of consciousness or with unpredictable rationality (children) drive.
Re:Open Problems in Robotics by YeGoblynQueenne on Wednesday August 19, 2020 @06:41AM (#24208974)
>> Humans don't have "error bounds" either, and you trust them just fine.

Yes, but humans are already here and doing all those dangerous and difficult things. But if we're going to replace them with something, it makes sense to want to replace them with something that's better than them. I mean us.

Otherwise, we might as well stick with the humans. Especially since we already know how to make humans (whereas self-driving cars, not so much).

Re:Open Problems in Robotics by Shared404 on Wednesday August 19, 2020 @03:19AM (#24207988)
I personally do not trust humans.

However, we do have inertia with humans running such situations, and until there is something provably/demonstrably better, I don't see the current situation changing.

Re:Open Problems in Robotics by imtringued on Wednesday August 19, 2020 @03:45AM (#24208117)
Trust is a continuum. I don't see how you can say that you do not trust humans at least to some extent, no matter how tiny that sliver of trust is.
Re:Open Problems in Robotics by Shared404 on Wednesday August 19, 2020 @12:20PM (#24212153)
A good point. I do of course trust humans to an extent, but as a general rule I feel that it's been beaten into my head by enough experiences to expect for humans to mess something up.

This includes myself of course.