News for hackers. Stuff that matters.
Re:Uber vehicle reportedly saw but ignored woman it struck by tlb on Monday May 07, 2018 @02:09PM (#17014954)
The disturbing thing about this crash is that it's such a basic thing to test on a test track. A pedestrian walking in front at night, with nothing else unusual happening, should be on the list of scenarios to get 5 9s reliability on before taking it out in public.

The things you have to test in public are the complex real-world interactions, like these: https://medium.com/kylevogt/why-testing-self-driving-cars-in...

Re:Uber vehicle reportedly saw but ignored woman it struck by ars on Monday May 07, 2018 @02:29PM (#17015141)
> get 5 9s reliability

That's it? Humans have greater than 7 9's reliability when driving. (As measured by fatalities per time of driving, and assuming a fatality takes 5 minutes, and average speed is 30mph.)

If the cause of a fatality is 10 seconds of failure, not 5 minutes, humans have greater than 9 9s of reliability.

Math: 10 seconds / (1.13 / (30 miles/hour / 100 million miles))

Fatality rate is 1.13 per 100 million miles.

Good luck programming a computer to even stay on and not crash with that level of reliability.

There's this belief that humans are horrible at driving, etc, etc. Until you run that math and realize, no, they're not. There's just a lot of driving going on.

Re:Uber vehicle reportedly saw but ignored woman it struck by jstanley on Monday May 07, 2018 @02:39PM (#17015222)
It's only really 9 9s of reliability if you assume somebody will definitely be killed as soon as you have a 10-second lapse in concentration, which is obviously not true.

In motorway driving you could probably shut your eyes for 10 seconds every 20 seconds and be perfectly fine almost all of the time. Especially at night time, or other quiet times of day.

Re:Uber vehicle reportedly saw but ignored woman it struck by ars on Tuesday May 08, 2018 @01:14AM (#17018957)
If people actually did that, then someone would crash, and that crash would be factored into the final accident rate.

So that final reliability rates includes any and all things humans do while driving. (Not what they could do, what they actually do.)

Re:Uber vehicle reportedly saw but ignored woman it struck by jstanley on Tuesday May 08, 2018 @03:07AM (#17019369)
Yes but what I'm saying is you don't need 9 9s of good driving in order to get 9 9s of accident-free driving. You only have an accident when you're driving badly and are exceptionally unlucky (I suppose the amount of bad luck required decreases as the badness of the driving increases).

In the scenario I mentioned you'd have at best a 50% duty cycle of good driving, but the accident rate would be substantially lower than 50% every 10 seconds.

Re:Uber vehicle reportedly saw but ignored woman it struck by erobbins on Monday May 07, 2018 @07:54PM (#17017519)
> There's this belief that humans are horrible at driving, etc, etc. Until you run that math and realize, no, they're not. There's just a lot of driving going on.

I've been hammering this concept for a while :) Unimpaired, non distracted humans are actually really good at driving. We're optimized for processing visual inputs and making fuzzy decisions, have high dexterity and coordination, and fear death. All of that is alien to software.

Re:Uber vehicle reportedly saw but ignored woman it struck by ufmace on Tuesday May 08, 2018 @12:35AM (#17018846)
Oddly enough, even impaired drivers are probably pretty good compared to some current self-driving cars. I'd give good odds that someone with a 0.10% BAC would have swerved or stopped for that woman that the Uber car hit.
Re:Uber vehicle reportedly saw but ignored woman it struck by coding123 on Tuesday May 08, 2018 @12:44PM (#17022352)
Yep sadly too the Uber car didn't even slow after the hit, so even a drunk person probably would have done different things... Enough maybe to be non-fatal. Not saying we should replace Ubers sdv fleet with drunk drivers obviously the entire program is scrapped now.
Re:Uber vehicle reportedly saw but ignored woman it struck by blattimwind on Monday May 07, 2018 @02:51PM (#17015337)
> If the cause of a fatality is 10 seconds of failure, not 5 minutes, humans have greater than 9 9s of reliability.

If you're driving by the textbook, then you have two seconds to react to the car in front braking; the assumption being that you take one second to react normally, and get one extra second for safety.

On the other hand, in something like a play street law grants you exactly zero seconds of response time.

The question is now, of course, whether only those <<10 s intervals count as the failure, or your entire approach to driving (e.g. driving while texting).

Re:Uber vehicle reportedly saw but ignored woman it struck by ars on Tuesday May 08, 2018 @01:24AM (#17018996)
> then you have two seconds

Which takes us to 10 9s of reliability. Are there any computers with that level of reliability?

> The question is now, of course, whether only those <<10 s intervals count as the failure, or your entire approach to driving (e.g. driving while texting).

I thought about this all day. I feel that from the moment you can no longer avoid to accident (no matter what you do), until it occurs, is the correct timeframe to measure. Not sure what that number is though.

Say you texted, got in a dangerous situation, then corrected. Is that really a failure? It's a risk of course, and enough people doing that will increase the final death tally. But for each individual driver it's not a failure, if it's not a failure then you can not count it.

i.e. doing it any other way would be counting it twice: Once for those situations that actually caused an accident, and again for taking the risk.

Or put another way, taking a risk and winning is not a failure. (Otherwise where do you draw the line on what a risk is?)

Re:Uber vehicle reportedly saw but ignored woman it struck by sdenton4 on Monday May 07, 2018 @04:07PM (#17016047)
Fatalities seems like a pretty low bar to measure failures by... I would prefer to measure reliability by reported accidents per 100k miles. This stat seems to be 183 per million miles in 2009, compared to 1.13 for fatalities per million miles. So quite enough to shave 7 9's down to 5ish...
Re:Uber vehicle reportedly saw but ignored woman it struck by oldgradstudent on Monday May 07, 2018 @11:51PM (#17018672)
The problem is that a lot of accidents are not reported and even injury severity can be very subjective (especially if there's an interest to do so[1]). I prefer counting fatalities because it's a hard endpoint - bodies are hard to hide.

[1] https://www.youtube.com/watch?v=_ogxZxu6cjM

Re:Uber vehicle reportedly saw but ignored woman it struck by cimmanom on Monday May 07, 2018 @06:10PM (#17016890)
Collisions with trees and with other vehicles? Or collisions with pedestrians? Maybe we're fine with five 9s of reliability with inanimate objects and with other 2-ton metal crash protection cages but prefer seven 9s with fragile unprotected human bodies (which are far more likely to end up as fatalities than just crashes).
Re:Uber vehicle reportedly saw but ignored woman it struck by ars on Tuesday May 08, 2018 @01:26AM (#17019010)
> So quite enough to shave 7 9's down to 5ish...

From other comments in this thread, the correct number is actually 10 9's, not 7. So using the accident rate instead of fatality takes you to 8 9's.

Re:Uber vehicle reportedly saw but ignored woman it struck by smogcutter on Monday May 07, 2018 @02:39PM (#17015223)
I agree with the basic point, but does the 1.13 per 100 million miles figure include freeway driving?
Re:Uber vehicle reportedly saw but ignored woman it struck by ghaff on Monday May 07, 2018 @02:50PM (#17015331)
Yes.
Re:Uber vehicle reportedly saw but ignored woman it struck by hsljekskfh on Monday May 07, 2018 @06:47PM (#17017100)
freeway driving causes less accidents than street driving
Re:Uber vehicle reportedly saw but ignored woman it struck by dllthomas on Monday May 07, 2018 @11:20PM (#17018567)
I think you were downvoted to dead because people saw this as an objection to the parent; more charitably, I assume you're just adding more color - but you might want to telegraph your intentions a bit clearer.
Re:Uber vehicle reportedly saw but ignored woman it struck by ars on Tuesday May 08, 2018 @01:16AM (#17018963)
He was not downvoted to dead, his account is banned, all his posts show up dead.
Re:Uber vehicle reportedly saw but ignored woman it struck by dllthomas on Tuesday May 08, 2018 @11:36AM (#17021686)
Gotcha
Re:Uber vehicle reportedly saw but ignored woman it struck by kbutler on Monday May 07, 2018 @02:22PM (#17015080)
5 9s sounds impressive, until you think that's 1 failure per 100,000 interactions.

There are individual intersections in the US that have that many vehicles pass each day.

I haven't seen estimates on number of specifically vehicle-pedestrian interactions, but 5 9s is nowhere nearly safe enough.

Re:Uber vehicle reportedly saw but ignored woman it struck by hmschreck on Monday May 07, 2018 @03:16PM (#17015587)
How many times does a car see this particular interaction, though?
Re:Uber vehicle reportedly saw but ignored woman it struck by bpicolo on Monday May 07, 2018 @04:51PM (#17016386)
It needs to handle new interactions too. Humans aren't predictable
Re:Uber vehicle reportedly saw but ignored woman it struck by Piskvorrr on Wednesday May 09, 2018 @04:35AM (#17028254)
That's also a factor. I've been driving for decades now, yet still need to handle surprisingly novel situations now and then.
Re:Uber vehicle reportedly saw but ignored woman it struck by digikata on Monday May 07, 2018 @03:31PM (#17015716)
I think the disturbing thing to me is that there wasn't any sort of layered reliability approach to the testing process itself. When you know you're going to be testing changes to a very complex, multi-sensor real-world system, it would be a standard approach to lay in a more foundational set of technology and practices to the system tests to serve as a conservative safety net.

For example, an overriding set of heuristics to brake if needed could be one precaution, and in terms of practices, the human monitor, should have been swapped out at high intervals to alleviate attention fatigue. One can go back and forth on the details, but the need for a separate and independent treatment of safety reviews from the primary product development activity in this kind of endeavour should have been drawn from any of a number of existing practices in other industries.

And this is something that a lot of the casual tech articles completely miss - criticizing the primary autonomy tech which of course will have faults, but if you know you will have faults, then where is the serious plan to operate safely knowing the dev system is anticipated to have faults of many types.

Re:Uber vehicle reportedly saw but ignored woman it struck by mnm1 on Monday May 07, 2018 @06:48PM (#17017109)
You mean they shouldn't have disabled the car's own autonomous, non AI braking system and kept it as a backup for when theirs failed? Yeah, but that would take a responsible, caring player and we're talking about uber here, a company built on ignoring laws, best practices, and common sense. This can easily be a case of manslaughter, but their cozy relationship with the az government will probably prevent that from happening despite uber's egregious recklessness.
Re:Uber vehicle reportedly saw but ignored woman it struck by digikata on Monday May 07, 2018 @07:22PM (#17017314)
> You mean they shouldn't have disabled the car's own autonomous, non AI braking system and kept it as a backup for when theirs failed?

I admit that was a thought, but I'd hold back from to saying that simple specific thing was the missed item, or even if that was a realistic expectation to maintain that setup. Without a built up safety culture that looks at the issue from multiple directions in detail, and has the authority to halt testing if needed, then it could have easily been something else. And there are multiple industries from civil/military aviation, medical, architecture, as well as many industrial occupations from which one could have formed a safety framework upon.

Re:Uber vehicle reportedly saw but ignored woman it struck by retSava on Monday May 07, 2018 @02:13PM (#17014995)
That, and the inattentive safety driver. It's quite infuriating really, letting lose a ton and a half of fast-moving mass without even the decency of looking where it's going. There's really no excuse.
Re:Uber vehicle reportedly saw but ignored woman it struck by CobrastanJorji on Monday May 07, 2018 @02:26PM (#17015109)
Unfortunately I think it's one of those things that is both inexcusable and guaranteed to happen.

It absolutely makes sense to put a human in a car you're testing and don't have full confidence in yet but are sending into real traffic. Not doing so would be ludicrously negligent. However, humans simply don't maintain focus for a long period of time without some sort of extreme, years-long training. The ability of monks to concentrate on a single thought for hours is correctly seen as very, very difficult. There's no way any regular person could deal well with a scenario like "sometime in the next 8 hours, you may or may not need to smash this brake pedal with a second's notice, but other than that you have no task." That's one of the things that most worries me about "almost there" systems like Tesla's.

That said, if you've decided to attempt this anyway, checking your phone is definitely not okay.

Re:Uber vehicle reportedly saw but ignored woman it struck by JorgeGT on Tuesday May 08, 2018 @04:34AM (#17019641)
> but other than that you have no task.

Then give them a task! Install a HUD drawing a box in real time with the classification of nearest detected object (bike, pedestrian, car, mailbox) and put buttons in the wheel for the driver to confirm/reject the classification. This way:

a) you improve your machine learning

b) driver has wheels on the road

c) driver has hands on the wheel

d) driver is focused on incoming objects

Re:Uber vehicle reportedly saw but ignored woman it struck by SilasX on Monday May 07, 2018 @02:52PM (#17015352)
I think there's some room for improvement though. Instead of "just one human in there", they could put them out in teams of two, and have periodic "awareness checks" that you have to respond to by pressing a driving-irrelevant button.
Re:Uber vehicle reportedly saw but ignored woman it struck by blattimwind on Monday May 07, 2018 @03:00PM (#17015436)
Dead-man switches do not work.
Re:Uber vehicle reportedly saw but ignored woman it struck by SilasX on Monday May 07, 2018 @03:12PM (#17015549)
Don't work for what purpose, according to whom? Are you sure that applies here?

What I suggested there isn't really a deadman's switch ("stop if no one is around to do task x"), but something to periodically, randomly check the driver's attentiveness, and can raise a red flag if they take too long to respond.

A deadman's switch is different and enforces "do X if person Y stops doing this thing at this predictable interval" -- not what I was proposing.

Re:Uber vehicle reportedly saw but ignored woman it struck by blattimwind on Monday May 07, 2018 @04:19PM (#17016152)
Vigilance control. But even these fail with more experienced users, similar to how alarm clocks fail some people (they wake up just enough to get up and turn the alarm off, then continue sleeping).
Re:Uber vehicle reportedly saw but ignored woman it struck by blattimwind on Monday May 07, 2018 @04:19PM (#17016151)
Vigilance control. But even these fail with more experienced users, similar to how alarm clocks fail some people (they wake up just enough to get up and turn the alarm off, then continue sleeping).
Re:Uber vehicle reportedly saw but ignored woman it struck by SilasX on Monday May 07, 2018 @04:33PM (#17016259)
I was proposing it as a way to validate engagement by the testing teams, not the mass market.
Re:Uber vehicle reportedly saw but ignored woman it struck by blattimwind on Monday May 07, 2018 @04:36PM (#17016284)
Like I said, these are especially hard to get right for frequent users, presumably your testing teams would log a lot of hours on these - just like train drivers, which have had this exact same problem.
Re:Uber vehicle reportedly saw but ignored woman it struck by SilasX on Monday May 07, 2018 @05:09PM (#17016520)
Yes, you said it several times, just not with a citation or other means of grounding your claims that might lead us to a way of resolving the disagreement productively.
Re:Uber vehicle reportedly saw but ignored woman it struck by baq on Tuesday May 08, 2018 @06:20AM (#17019949)
they do but they must be very random so you can't activate them when asleep.
Re:Uber vehicle reportedly saw but ignored woman it struck by Rjevski on Monday May 07, 2018 @02:49PM (#17015322)
However, if the car saw the pedestrian, even as a false positive (presumably from far enough away to have enough time to brake/avoid it), I think it should’ve alerted the driver. The driver would’ve most likely started paying attention and avoided the crash.
Re:Uber vehicle reportedly saw but ignored woman it struck by dllthomas on Monday May 07, 2018 @11:27PM (#17018591)
> I think it should’ve alerted the driver.

I like the idea, and if these pruned false positives are rare enough I think it would be a great safety measure - you could even have the driver verbally label things, which might have both training (in the AI sense) and awareness (for the humans) benefits (https://en.wikipedia.org/wiki/Pointing_and_calling).

Note that "rare enough" needn't be all that rare, just that if it is multiple every second or something it clearly wouldn't be practical.

Re:Uber vehicle reportedly saw but ignored woman it struck by mehrdadn on Tuesday May 08, 2018 @04:11AM (#17019578)
>> I think it should’ve alerted the driver.

> I like the idea

I'm not sure how well I would respond to a sudden alarm when everything has been deemed safe until a moment ago. I'd potentially need several seconds to figure out what is going on.

I would hazard a guess that an approach that might be more fruitful be to have a meter that dynamically adjusts the "riskiness" of the situation between a yellow, orange, and red zone, so that the driver (a) has to pay attention constantly, (b) gets information constantly, and hence (c) has a better chance to react earlier. I know if I see a "danger meter" getting into the orange zone, I'm not going to wait until it goes into the red before I start paying attention.

Re:Uber vehicle reportedly saw but ignored woman it struck by dllthomas on Tuesday May 08, 2018 @11:46AM (#17021800)
"I like the idea" doesn't mean I will like any given implementation of the idea - I certainly agree that there's plenty of room to get the interface wrong.

I want the driver paying attention constantly, but I want them paying attention to the environment, not a meter. A periodic "things are unusually interesting, what's going on?" query seems like a way to motivate that. But any actual attempt at a solution should be validated in testing...

Re:Uber vehicle reportedly saw but ignored woman it struck by taneq on Tuesday May 08, 2018 @02:56AM (#17019322)
GM SuperCruise uses a driver-facing camera to confirm that the driver does in fact have eyes on the road. Something like that would be trivial for Uber to add to its self-driving car fleet.
Re:Uber vehicle reportedly saw but ignored woman it struck by Lionsion on Monday May 07, 2018 @03:30PM (#17015708)
> However, humans simply don't maintain focus for a long period of time without some sort of extreme, years-long training.

And anyone who's achieved this ability is unlikely to be willing to be hired as 1099 contractor for whatever low pay is being offered.

Re:Uber vehicle reportedly saw but ignored woman it struck by Piskvorrr on Wednesday May 09, 2018 @04:39AM (#17028267)
Immaterial. "Let's keep killing bystanders, that's cheaper" is a good way to rouse a mob with torches and pitchforks; conversely, not very useful for getting the technology approved.
Re:Uber vehicle reportedly saw but ignored woman it struck by gizmo686 on Monday May 07, 2018 @02:27PM (#17015116)
Humans are terrible for supervising mostly functional machines.

This was researched in the context of security guards, who had a 95% missrate after just 20 minutes on the job.

It is entirely unreasonable to expect a human supervise to detect and correct this type of situation in time.

Re:Uber vehicle reportedly saw but ignored woman it struck by syncsynchalt on Monday May 07, 2018 @02:34PM (#17015195)
I both can and can't blame the safety driver.

To expect full attention from a person that is not supposed to have anything to do for hours on end (different from driving, which requires constant inputs and adjustments) is not something that's compatible with humans.

With that said they were looking at their phone instead of zoning out eyes front.

Re:Uber vehicle reportedly saw but ignored woman it struck by trgn on Monday May 07, 2018 @02:32PM (#17015172)
yes for sure.

I wish articles about this crash would continue to hammer on the basics, to keep front&center this crash was completely avoidable.

Even if there were no technical issues,

(1) the driver was on her phone before and during the crash. The whole reason for the driver to be present is to intervene in a situation like this.

(2) the car was driving too fast for conditions. Nobody should drive this fast in the middle of the city in the middle of night. It doesn't look like the car could have come to any kind emergency stop. Self-driving cars should drive more carefully than humans, not push against and over the speed limit regardless of outside conditions.

Re:Uber vehicle reportedly saw but ignored woman it struck by Aspyre on Monday May 07, 2018 @08:59PM (#17017925)
It was 38 mph in a 45.
Re:Uber vehicle reportedly saw but ignored woman it struck by lewiseason on Tuesday May 08, 2018 @05:56AM (#17019858)
The overriding principle is normally about driving to the conditions, not achieving a "target" speed.
Re:Uber vehicle reportedly saw but ignored woman it struck by Piskvorrr on Wednesday May 09, 2018 @04:47AM (#17028307)
This is one of the core problems of the car culture: ignoring the envelope except where mandated by law. A speed limit means, literally, "you are not allowed to go any faster than this" - but is interpreted as "you are supposed to go at this speed, or better 5 above". Nowhere else is this so prevalent: "there are 50 apples here, so you can't eat any more" is "you must eat all 50 of them" by the same logic.