If that's the best Uber can produce then they ought to hang their heads in shame. Unless it was doctored with... as I find it hard to believe they'd put such rubbish quality cameras in their trials.
Do you trust Uber to provide unedited raw video, or would they process it to increase contrast, make it appear that nothing was visible in the dark areas of the frame, reduce the resolution, drop frames, etc.?
Edit: Nevermind. Someone posted a picture of the car's interior, below and there's no computer screen.
https://arstechnica.com/cars/2018/03/police-chief-said-uber-...
Please check the videos out.
Is that the same cam used by the AI to detect obstacles?
I would expect a safe self driving car to include IR cameras that can be more cautious about moving warm blooded creatures.
Surely some more detailed telemetry data would reveal whether the main issue is with the sensors or with the algorithm.
Both of those have eyes that act as reflectors and you can see their eyes well before you can actually see the whole animal.
This[0] suggests that the total time required for a human to avoid an incident like this is 3.6s (at 35 mph, casual googling suggests the car was doing 40). Even if we add 1 second of extra time to deal with it I'm not sure that makes the cut.
Also remember she was not a stationary object. She was in the act of crossing the road. Human eyes/brains are good at detecting motion in low light even if we can't 100% make out what the object is.
I have lived in Tempe and know that part of town well. There are apartments, gas stations, hotels, strip malls, fast food restaurants and a strip club. It's not a pitch black country road.
Your last paragraph is a valid calculation if this were a case of a person stepping directly off a curb into the lane of traffic. However, it appears that they were probably standing on the median looking to cross, then stepped off into the left-most lane of traffic, an empty lane, proceeded across that lane towards the lane in which the car was traveling. In this sort of situation human intuition will recognize that a person standing on the median of a high-speed highway is likely to do something unusual. Particularly when you observe the visual profile of, as media has reported, a homeless person who is using the bicycle with numerous plastic bags hanging off it to collect recycling.
Also, has anyone here talked about the effect on the eyes of watching a (typically) bright white screen vs letting them adjust to the light of the night yet? This point deserves to be brought up.
Perhaps the video was intentionally darkened to simulate this effect. :P
Using bright interior lighting at night is something that we've known not to do for more than a century. If the driver couldn't be expected to see the pedestrian because the interior lighting or UX was too bright that is not something that does not reflect favorably upon Uber.
If so (a hand held phone), in Australia that driver would be going to jail for culpable driving causing loss of life.
EDIT: Upon re-watching the video a third time and really paying attention to this I don't think there is any real way for us to know without confirmation from the driver them self or an official report on the incident. My mind was definitely deciding things that just aren't discover-able from the video itself.
"Uber also developed an app, mounted on an iPad in the car’s middle console, for drivers to alert engineers to problems. Drivers could use the app anytime without shifting the car out of autonomous mode. Often, drivers would annotate data at a traffic light or a stop, but many did so while the car was moving"
https://mobile.nytimes.com/2018/03/23/technology/uber-self-d...
The whole project seemed designed for an outcome like this. Eg allowing app to be used whilst on the move, after reducing from 2 to 1 operators. Culpability ought to lie with Uber.
https://cdn.geekwire.com/wp-content/uploads/2018/02/Front-_-...
A different picture from that article shows that under the GPS is the gear stick, an emergency button, and a cellphone charger.
Autonomous cars are never going to be viable. Just looking at the cost of high-end SLR sensors and lenses that you'd need to match human eye dynamic range, and you're already looking at an expensive setup, before we even get to things like 360-degree vision and IR/LIDAR/Hyperspectral imaging. And that's in addition to all the compute problems.
Sorry Silicon Valley tech-bros, but it's a fantasy you're chasing that's never going to happen. The quicker we can end this scam industry, the better.
People really need to be told "no".
Probably better to chase after flying cars..
They should pivit to augmenting drivers, not attempting to drive for them. I would happily utilize a properly designed HUD (meaning I have source access) connected to a fast MerCad or bolometer array.
But you are right, though. I think augmenting drivers sounds like a great idea in the sense you talk about. The kind of augmenting drivers I don't want are those stupid headbands you'd wear that beep like crazy if your head starts tilting in a way that resembles falling asleep. If you are in danger of falling asleep at the wheel and need a device like that I think it's pretty obvious one should take a nap on the side of the road or in a free parking lot, haha. Hopefully if we do wind up headed in that direction the people inventing will have a similar way of thinking and inventing.
It really is. The eye can detect a single photon. Fingertips can detect 13nm bumps, smaller than a transistor on Coffee-lake CPUs.
We're better off acknowledging machine limits to work on other problems instead.
No. A camera's dynamic range is pretty much fixed. If you can capture low-light objects it means high-light objects are completely blown out.
The Canon ME20F-SH is a video camera reaches ISO 4,000,000. This camera has a dynamic range of 12 stops and is available at B&H for $20,000. [4]
Of course, this isn't exactly the challenge that cameras face when assessing a scene. The dynamic range happens within a single scene all at the same time. Wide dynamic range (WDR) is the term I've seen used in describing video cameras that can handle both bright and dim areas within the same scene.
[1] http://lightartacademy.com/blog/tutorials/camera-vs-the-huma...
[2] https://www.youtube.com/watch?v=ZRzXgSMbBu0
[3] https://www.cambridgeincolour.com/tutorials/dynamic-range.ht...
[4] https://www.bhphotovideo.com/c/product/1187825-REG/canon_100...
There are a number of video samples shot on the Canon ME20F-SH on YouTube. In these one can see that under low light situations the camera is shooting at ordinary video speed (the camera supports shutter speeds from 24 to 60 fps). I'm not trying to push the Canon ME20F-SH; I don't have any association with Canon. The manual for this camera is available on-line if you'd like to read up on it: [1].
The actual exposure of a video frame or image depends upon the f-stop of the camera's lens (aperture), the shutter speed, and the ISO of the image sensor. See [2].
Basically, each doubling or halving the shutter speeds corresponds to one "full-stop" in photography. Each full stop of exposure doubles or halves the amount of light reaching the sensor. Changing the aperture of the camera's lens by full stops also doubles or halves the amount of light reaching the sensor. Full stops for camera lenses are designated as f1, f1.4, f2, f2.8, f4, f5.6, etc. The light sensitivity of the film or sensor is also customarily measured in full stops. Very slow fine grained color film is ISO 50 and is usually used in full sunlight. ISO 100 is a bit more flexible and ISO 400 used to be considered a "fast" film for situations where more graininess would be acceptable in exchange for low light situations. Each doubling of ISO number corresponds to a full stop. So a photo take with ISO 400 at f2 with 1/1000 second shutter would have the same "brightness" as a picture taken at ISO 100 at f2.8 with 1/125 second shutter (less 2 stops ISO, less 1 stop aperture, and plus three stops shutter speed). Naturally, other factors come into play, the behavior of film or digital sensors at extremely slow or extremely fast shutter speeds isn't linear, there are color differences, and noise issues too. See [3] if you are interested in more about how photography works.
[1] https://www.usa.canon.com/internet/portal/us/home/products/d...
[2] https://photographylife.com/what-is-iso-in-photography
[3] https://www.amazon.com/Negative-Ansel-Adams-Photography/dp/0...
2. I'm not sure you can fix the exposure on "fairly cheap" dashcams.
3. "high-light" in night settings are likely much lower than standard daylight, let alone bright daylight.
Though I guess you could have an auto-exposure dashcam standard and add a low-light one which is only active in low light conditions.
Now that I think about it, self driving cars may be paralyzed by other self driving cars running IR boosted headlights.
Waymo cars are capable of sensing vehicles and pedestrians at least half a block away in every directions. I was reserving any judgement on wether this collision could have been prevented, but seeing the video tells me that 1) a human driver might have hit the victim regardless, and 2) I'm very surprised that the LIDAR sensor didn't cause the car to stop to a halt much, much earlier. This is exactly the kind of situation that I would expect self-driving cars to be better than human drivers.
What you posted looks pretty cool, I don't know enough about it to understand what I should be prioritizing focus on, but we can chalk that up to ignorance. The benefit that driver-view footage has is that it is a viewpoint all of us are familiar with. If you ask me to watch dashcam footage to assess some kind of traffic thing, there's a general expectation of where I keep my eyes and what I notice.
This normal-human-view mode is probably going to be necessary in AV cases in which we determine whether the car's AI did the right thing. Presumably, as AV becomes mainstream and extremely safe, these accidents will involve edge cases and outliers which are poorly interpreted by sensors/non-human-vision. Seeing the scene as a human driver does might be a necessary starting place?
But the Uber case in AZ, IMO, proves your point. The Tempe police quickly made a judgement call based on what seems to be inadequate video. Everyone who can now view the video will also be inclined to think how impossible it would be to avoid hitting the victim, even if the actual scene in-person has much more light. And of course, we don't want to judge AV solely on whether it performs as well as normal humans.
You don't think performing as well as normal humans should be sufficient to allow them on the road?
Or are you saying they should be allowed even if their performance is worse than human? (...as long as some other criterion is met?)
Even if the camera was brighter uber isn't at fault anyway....
However, as a society and civilization, and even more so, as engineers and scientists, we are going to expect that the autonomous car matches or exceeds human-level performance in critical situations like this.
Therefore the time spent on investigating, understanding, and discussing the root causes of the accident is worth understanding. Accidents like these generally do not happen due to a single factor. It is necessary to understand all the necessary factors if we want to make autonomous driving systems more reliable.
At the very least we need to understand whether the pedestrian appeared in the other sensors that a human could have identified by looking at the sensor data, and if yes, whether the autonomous system matched or exceeded human-level performance by detecting the pedestrian, and if the pedestrian was indeed detected, why the autonomous driving system failed to respond to the situation.
However, most auto liability insurance covers whoever you permit to drive the vehicle, so the owners policy does typically cover the fender bender on the way to the grocery store.
I would be very interested to learn whether or not the car's autonomous system identified a bicycle at any point prior to the collision.
car didn't see it at all even in those last moments.
But this was a pedestrian, not a cyclist.
Personally, while riding at night, I look like a Christmas tree. $10 on EBay goes far these days in the reflective tape and bike light department:
What is surprising is that the bike didn't seem to have Tire reflectors like these:
https://www.wired.com/2011/11/fiks-reflective-rim-strips-for...
They are mandatory in lots of countries, to the point that it's impossible to buy tires without them. All brands come with them.
Maybe the outcome will be that thermal infrared will be mandated on all sensor packs?
I just watch the road in front of me.
My idea is that the car has been behaving well for a long time and consequently the driver lowered is vigilance. Big mistake.
Unlike many others, sadly - even when they don't have any self-driving tech at all
This was purely bad software, and no failure scenario being programmed in. I really don't think it's that difficult to program split-second reaction to obstacles that appear into the driving path. We need to get to a point where these vehicles can do stuff like this, even in a 2-dimensional way:
https://nacto.org/docs/usdg/relationship_between_speed_risk_...
Getting hit at 10mph still is going to suck but it’s a lot more likely to be broken bones and road-rash.
They seem to use 2.5 seconds as the standard for drivers to perceive and react to an obstacle, which based upon studies covers 90% of all drivers. 1.5 seconds to perceive, 1 second to react. Then you have maneuver time on top of that 2.5 seconds.
Given this, 1 second seems very low. A large percentage of drivers would probably plow into them at full speed.
If all but the slowest 10% can react in 2.5 seconds than I would think many would do a fair bit better.
Edit: Apparently the average person is closer to 1.1 seconds. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.372...
Someone else thought the same thing and went to get their own footage of the road.
In general, traffic safety requires that road planners ensure that one of three conditions always applies:
a) the roads are lighted from above; b) cars are able to use high beams; c) there are no pedestrians crossing the highway.
This can be done in general, mostly by investments in infrastructure to ensure lighting or isolated highways wherever the density doesn't allow to drive with high beams.
Seriously, what else can you expect. These companies who do put these things on the road with the justification that "There is a human behind the wheel" should be taken out back and shot in the head...Just pull the plug. No more self driving cars for them. Those are just the kind of tech companies we don't want around...
See, it is not a mistake that they are making. They know well enough that this human behind this wheel is a useless as a dummy. But they do it any way. What does it say about them?
I see waymo drivers all the time actively paying attention to the road.
2. These companies seem to be doing nothing to make sure that the drivers will pay attention always and is always in a position to intervene. They even seemed to allow smart phone usage while they are in the car.
So, according to them, the human behind the wheel is just a decoy to prevent backlash from officials and the public, so that they can always say, "look, there is a human behind the wheel if something goes wrong"...
Also, even if they implement some measures, they can only make sure that the driver has eyes on the road. Not that they are actually paying attention. A driver who is actively driving the car will notice a lot more stuff than a passenger who is just looking at the road. There is no way to make a human pay that kind of attention with out actually driving the car. So at best, your "driver behind the wheel" is as good as a passive passenger.
And as told before, the companies are not even trying to make sure of that.
In that context, the landscape changes significantly - instead of a self driving car that mowed down a pedestrian, we have a driver who was too busy looking at her phone to pay attention to what her vehicle was doing. From the various articles, it seems that she's not an engineer, and is there in effectively the same capacity as any other Uber driver. If that's the case, she's putting far too much trust into an experimental system. I agree that Uber could do more in the way of technological means to ensure the driver is paying attention, but at some point, an adult with a job needs to be responsible for doing that job.
The framework should have been in place before these vehicles were ever put on the roads. For example, there should have been some formally specified tests for a self driving vehicle before it can be put be on the road, even with a back up driver..
> a fully autonomous car is legally no different to cruise control - it's just a driver assist, and the human behind the wheel is still ultimately responsible for whatever the vehicle does.
Any thing that does not require drivers to keep their hands on the wheel is not a driver assist. It IS the driver. So there should be tests that make sure of the competence of the tech that is in the drivers seat.
I don't know how people let this happen!
I'm certain that if you can design and build a self-driving car that you can design a simplistic human attention monitoring system that will cause the car to pull over if attention level is too low.
Gaze monitoring that checks for looking downwards or away from the carriageway for extended or too often repeated periods wouldd probably be enough.
I imagine the attention of the "vehicle operator" is vital to the proper training of the vehicles -- if they don't see near misses, or failures to slow for potential hazards, or failures to react to other road users then how can the softwares faults be corrected? Do they get a human to review all footage after the drive?
And that’s consumer drive assist tech, not “we are experimenting with full autopilot” tech, where I’d think such safety measures would be even more appropriate.
This is a solvable and solved technical challenge. Uber just didn’t devote any resources to it because they don’t appear to give a shit beyond acquiring a legal fig leaf to shift liability from themselves to an individual.
I did wonder if you could require the driver to make control inputs that aren't actually used to control the car but are monitored for being reasonably close to how the computer is controlling the car, and then the automation disengages (with a warning) if the driver is not paying sufficient attention. I then realised that may be _worse_ - in the event of a problem, the driver would have to switch to real inputs that override, which may delay action and not be something they do automatically. It would mean they are paying attention more to see if the automation is making errors where they have more time to react though (e.g. sensor failure that is causing erratic behaviour but not led to an emergency situation).
I wonder if a hybrid approach might be viable -- fake steering is used to ensure that the driver is alert and an active participant, but the driver hitting the brakes immediately takes effect and disengages the automation.
Looking forward to seeing this play out in court.
Wait, aren't you mean to have your hands on the wheels at all times? I don't see what to be skeptical about when if he just followed the law this could have been avoided.
It seems to me the driver might be in for some legal trouble.
Probably checking the computer installed for diagnostics of the autopilot system. If it's in self driving mode and you are the engineer in charge, you'd want to constantly check what the system is seeing vs the actual conditions on the road.
To me it looks like the guy is just falling asleep at a boring job. In all likelihood that was not an engineer more than any other taxi driver is an engineer.
The software is the "driver" of this car. Not the human behind the wheel. Take a look at job descriptions [0] for this. They always include a bit about "operating in vehicle computers". The fact, we don't know what the person is doing.
0 - https://www.indeed.com/viewjob?jk=597616bf7d02d899&tk=1c96sl...
I don't know about current regulations. Are companies now allowed to operate autonomous cars without a driver that pays attention?
People can safely drive in total darkness with the aid of their amazing human eyes and high-beams.
If for some other reason visibility is low you slow down - not rely on glancing at a backlit display ruining your own night vision and taking your eyes off the road for seconds at a time.
Or swerve out of the way.
It is surprising to learn that these vehicles are operating at night. To collect training data, since nighttime driving is inevitable, perhaps there are ways to simulate night to the computer vision systems during daytime so the human supervisor can still see clearly.
Would you trust this system that didn't even manage to slow down at all with a pedestrian slowly pushing a bike directly in front of it, artificially adjusted to be even worse, driving during the day??
I wouldn't.
> "The driver said it was like a flash, the person walked out in front of them," Moir said, referring to the backup driver who was behind the wheel but not operating the vehicle. "His first alert to the collision was the sound of the collision."
> "The driver, Rafael Vasquez, 44, …"
https://www.bloomberg.com/news/articles/2018-03-20/video-sho...
EDIT: what i meant, in light of the downvotes is that humans can train themselves to see, and just that folks driving in Asia have heightened sense of alertness, due to their environment. Hope it came out alright.
I'm reluctant to infer exactly what a human eye would have seen in that situation. I have absolutely driven down streets in suburbia where the gap between street lights was large enough to make them quite dark, and that video was an example of exactly what I was afraid of happening whenever I drove down those streets (though admittedly my fear was hitting a white tailed deer).
I think it might also be fair to argue that the car's high beams were not on (but again, that shouldn't matter because of LIDAR, right?).
I'm not confident even an above average human driver would be able to avoid that accident, even if good eyesight gave you an extra half second to respond. Dark clothing and no reflectors means that person was definitely invisible to both the camera and the driver for some time after they would have been visible in daylight.
I've had a couple of situations where someone appeared close to my line of travel with low visibility clothing (at night) that scared the living shit out of me, and they weren't trying to cross the street.
To be clear, I am not blaming the victim here, but do wear high visibility clothing when you're a pedestrian near high speed roads at night.
A similar thing (no fatalities, just a shopping cart pushed by homeless people) happened to me. Ever since then, I have learned to be much more aware of situations like this (tunnel of light surrounded by darkness).
This just shows that Uber's tech is bad and that they let it on the road shows that their culture is still at least partly rotten.
I don't think Dara is the do gooder that some people are making him out to be. His primary motivation seems to be to usher Uber to an IPO. IMHO, if he actually had ethics, he would be front and center on this. Your company just killed someone. Where are you?
Amusingly, the law also says that manufacturers have to produce headlights that cast light out far enough to leave you adequate stopping distance at 60mph. Almost no headlights on the market currently do that.
Not a counterpoint, just a tangent that I find sadly amusing.
The seemingly random design decision of many runner manufacturers to embed tiny reflector strips in their shoes have no doubt saved countless lives. And their owners would probably be none the wiser.
Of course it makes sense there where daylight may be hard to find half the year, however even in Australia, once it is dark the darkness is the same.
And I haven't seen a single government initiative to increase visibility awareness - most people are completely in the dark. (Sorry)
Riding shared bike trails in Melbourne at night on the commute home, this is something I think about often in the "winter" months. Peds may hate the strong glare from my LEDs, but it is the only thing the has half the chance of making out ninjas against the frequent sports ground stadium floodlights the path goes by.
That is not the whole reason, it is one of many reasons.
> To be clear, I am not blaming the victim here, but do wear high visibility clothing when you're a pedestrian near high speed roads at night.
Yes, stupid homeless person.
For anyone who's interested, try taking your phone with the camera app open into a dark room and comparing what you see to what's on the screen. Which shows more detail?
But the gap between street lights is going to be very hard to see into.
> I'm confident my eyes are good enough that I would have been able to see this person at night in these lighting conditions.
I think you're overconfident. Human low light vision is very good if there is low light everywhere. But it is not good at seeing into low light regions when brightly lit regions are nearby.
That said, I agree that a visible light video camera is likely to be even worse that human vision under the given circumstances. But as others have commented elsewhere in this thread, the car is not just supposed to be using a visible light video camera. It has LIDAR and IR sensors, which should have clearly shown the pedestrian well before visible light did.
In this video [1] driving northbound, same as the vehicle in the crash, the car first goes under AZ-202, emerges under a streetlight, goes through a darker spot, then another streetlight (as you see the rocky outcrop), and then a very dark spot: and suddenly, you see a right-turn lane that wasn't there before. The latter dark spot is where the crash happened.
Another video by the same author, driving southbound [2], provides another useful reference. And these videos are three years old, yet the illumination of the roadway has not improved. Cameras exaggerate the contrast a bit, but not unreasonably so. The streetlights in question essentially aim directly downwards, illuminating the roadway immediately underneath, but much less of the surrounding air than other designs. This is responsible for the dark gaps, albeit it does significantly reduce light pollution.
[1] https://youtu.be/zEaTdYJExq8?t=8m50s [2] https://youtu.be/yfR7krN7z00?t=23m26s
EDIT:
Found more. The car in this video is going southbound, camera facing backwards [3]. This view faces the same way as the Uber did, but of course this video is moving away from the scene, and offset by a few dozen meters to the west. The drastic change in roadway illumination can still be seen.
In a fourth video [4], the car is going northbound, like the Uber, in the proper lanes, but the camera is pointing obliquely front-right. The illumination seems better, but you can still see the intensity of the shadows, including environmental shadows and the car's own shadow, as it moves between the lights.
[3] https://youtu.be/0Dum8Fj71JU?t=13s [4] https://youtu.be/6qHcuW_LCIU?t=16m45s
Everyone is moaning and slicing and dicing what the self-driving vehicle did wrong but, since you're familiar with the area: are pedestrians typically expected to be crossing this road?
Seems like the accident has a lot of factors that might not only be the self-driving car's fault, nor even a human driver that was fully in control. Regardless of how well people may want self-driving cars to do, one thing that can actually exist in the present is to make sure that we are creating safe ways for pedestrians to cross a road.
I've driven many thousands of hours at night and have dealt with a fair number of crazy pedestrians including a rather ... uncoordinated ... guy in Casa Grande who decided to go in circles on his bike in the middle of the road at around 3 AM for no discernible reason. Fortunately that place was much better lit and I was able to see him and stop until he got out of my side of the road.
So it's not that common, but yes, every so often you will see some person in black jaywalking across a wide road at night and they're quite hard to see. I don't think a lot of people appreciate that the streets here are wide & fast and that there just isn't that much pedestrian traffic even in daytime.
Here in LA, it's dense and traffic can't get up to very high speeds and we have relatively frequent places to cross safely if people choose to do so. I've definitely seen those who choose not to walk an extra 100 feet to wait at a crosswalk nearly hit in dusk or night traffic.
No amount of automation is going to bring the accident rate down to 0 so through a combination of factors, such as traffic and community design, we can work in tandem with automated driving to get closer. There's still the X factor of our human ability to do really dumb stuff.
I've crossed these myself, but I always look both ways.
Tucson does it due to the nearby observatory. The greater Phoenix area has a huge glow that washes out all the stars. You can see the glow as far away as Casa Grande when you come out of the little rocky pass on I-10 north of there.
Edit: removed a duplicated word.
I live in the Phoenix area, on the west side closer to Glendale (specifically, the border between Phoenix and Glendale is literally in my back yard).
There are times in the summer where the glow from the city is so bright, that rather than a dark sky (never black), you have a grey dimly lit sky instead.
Literally, "the sky was the color of television tuned to a dead channel" - maybe not as bright as the static Gibson was referring to, but still bright enough to see by - even without a full moon.
This wholly contradicts my experience driving at night on a street with street lights. I can't recall a time in my entire life I have had significant difficulty seeing into the gap between street lights. Keep in mind that the gap is not arbitrarily chosen.
Edited to note that I have experienced difficulties in low-vis conditions such as snow storms, sand storms, VERY strong rain storms, etc. None of which apply to this situation.
I’m not sure a typical eye exam checks for it, either, because none of the tests I can think of seem like they’d be useful.
(As usual, an even keeled comment based on family experience is -2 and rapidly being silenced with zero feedback inside 5 minutes, which makes me wonder why I contribute to this community at all, probably time to stop)
I did one at my last eye exam and it was pressing a button when you see dim flashes in all different locations. If you had low sensitivity, you wouldn't see those flashes and presumably you'd get a low score.
I would also not judge the community based on reactions to this very contentious thread - i am wary of jumping in on this one, but thought it worth noting your comment was not wildly out of place.
stay, we have cookies :-)
I actually believe i am representing HN as a place where different opinions can be voiced, hopefully in a manner to generate light not heat. Heated discussions are rarely the useful or interesting ones to read.
Thank you for appreciating my enthusiasm.
PS Are you using two accounts - one ("my 'unpopular opinion' account) for saying things you fear people might not like? That seems odd. May I ask why?
It's not supposed to be, no. But the gaps are not always optimal. The spacing of the street lights in the video (to the extent I can tell) seems to be quite wide, wider than I would think is optimal.
Not according to the post by niftich upthread.
Looking, right now, at a parking lot between two lights from a well-lit room. I can make out most of the outline of the black car in the middle of the "darkness" without any trouble. This isn't even the low light vision kicking in (which I agree isn't going to kick in if you're driving). Human vision should be able to make out the pedestrian earlier than the video footage.
Also, are you looking straight at the car? Or are you looking elsewhere so that the car is in your peripheral vision, the way it would be if it were on the side of a road you were driving on?
I've personally driven down country roads without any lighting except my headlights and saw deer poking their head out of the woods a ways away for which I slowed down in case they darted across the street. Someone slowly walking their bike would be trivial.
Reminds me of this video of a Rally driver racing with malfunctioning lamps https://youtu.be/HwyRS_6Uqn0?t=2m36s
The video makes it seem impossible but afterwards in the interviews the driver said it wasn't too bad after his eyes adjusted. He did have some issues with his own lamp blinding him which lead to errors. (He actually won this stage.)
As far as I'm concerned Uber's software/hardware is completely at fault and not ready for public testing. I'm uncertain how much better everyone else's tech is but Uber's typical carefree approach has ruined it for everyone.
There are consumer level dashcam that can shift up to 12800 ISO which can create a fairly distringuishable picture with ambient moonlight.[1]
Canon builds sensors with ISO's in the millions which should be able to see distinguishable shapes without ANY light. [2]
[1] https://youtu.be/hHU-hWG5DDk?t=5m48s [2] https://petapixel.com/2015/09/13/this-is-iso-4560000-with-ca...
The headlamps may have been functioning, but they appeared to be aimed way too low. You can see that the car is able to traverse the distance lit up by the headlamp in about a second at 38 mph. If the headlamps were aimed properly, it should light up the road about 5 seconds ahead of the car.
A system with this rule baked in would be driving slower.
People adjust the way they drive based on what their environment is doing, how well their equipment is working and their own alertness. Except in the extremes we should not accept misconfigured equipment as an excuse. And if a system detects that there is no acceptably safe speed for it to go then it should not move at all.
Arguably, the system should detect a misconfiguration like this when the car is turned on and not allow the car to be driven until the problem is fixed.
They didn't. Either because they are negligent or because the footage is misleading and the system saw the whole thing but did nothing.
We typically cant see much detail in the scene out of our small region of focus, but you can bet if a tiger appears from behind a tree our visual system will scream to the brain _look over there right now!_
Our eyes and our entire visual processing system is very much not "just like a webcam, but made out of meat".
I have a dashcam, and I've seen night videos from it.
In fact, the picture from my dashcam seems much better than this low quality mess, but still night videos from it come out much less visible than reality.
I've tried to rewatch some parts of videos later, and I find I was able to see much more detail on the sidewalk and on the periphery than was captured by the dashcam. Everything gets blown out in the night videos by the headlights.
http://velodynelidar.com/hdl-64e.html (note that this LIDAR is expensive - costs way more than the car it is mounted to)
Here's the manual for it - note the specs in the back:
http://www.velodynelidar.com/lidar/products/manual/HDL-64E%2...
It has 64 lasers, spread out over about 27 degrees - about 0.4 degrees per laser, from almost horizontal to an angle of 24 degrees or so down. Now take a look at where it is mounted on the car, and envision these laser beams spreading out and being spun in a circular conical area around the car.
Now - if you think about it - as the distance from the sensor increases, the beams are spread further apart. I'd be willing to bet that at about 200 feet or so away from the car, very few of the beams would hit a person and reflect back. Also - take a look at the reflectance data in the spec. Not bad...but imagine you are wearing a fuzzy black jacket on your top half. How much reflectance now?
What do you think the point cloud returned to the car is going to look like? Will it look like a human? Hard to say - but you feed that into a classifier algorithm, there's a possibility that it's not going to identify the blob as a "human" to slow down. Especially when you add some bags, a strange gait, plus the bicycle behind the person. All of this uncertainty adds up.
I am also willing to bet that only the LIDAR was used for collision detection (beyond the radar on the unit). Any cameras - even IR based - would likely only be used for lane keeping and following purposes, plus traffic sign identification. Maybe even "rear view of vehicle" detection. Ideally it would be used for "person/animal" identification and classification to - but again, given the camera sensor, and who knows what the IR sensor saw or didn't see, along with the weird lighting conditions - well, who knows how it would have classified that mix?
Lots of variables here - lots of "ifs" too. All we can do is speculate, because we don't have the raw data. Uber would do well to release the entire raw dataset from all the sensors to the community and others to look over and learn from.
Finally - I am not an expert on any of this; my only "qualifications" on this subject is having taken and passed a couple of Udacity MOOCs - specifically the "Self-Driving Car Engineer Nanodegree" program (2016-2017), and their "CS373" course (2012). Both courses were very enlightening and educational, but could only really be considered an introduction to this kind of tech.
Certainly better than any camera mounted on a dashboard.
It's honestly a bit surreal how the pedestrian appears out of the splotch of pure darkness in the frame. That's low dynamic range and resolution (or high compression) at work, not how light behaves in reality.
https://i.imgur.com/AlO4h7p.gifv
I figured that light in front of the car was mostly just messing with the camera but that driver sure didn’t see that pedestrian either. I’m willing to give a human driver the benefit of the doubt here and say that even with eyes on the road and hands on the wheel the outcome would likely have been the same. The pedestrian was not highly visible - no reflectors, dark clothes, it’s really hard to see people like this.
unless their phone was showing video of the road in front of them, I don't know why they would have seen her.
The eye can gain a lot more stops through adaptation (irising, low-light rod-only vision), but those mechanisms dont come into play when viewing a single scene -- and cameras can also make adjustments, e.g. shutter speed and aperture - to gain as much, if not more, range.
I'm concerned about poor scotopic adaptation due to the rather bright light source inside the car - maybe it's the display he's looking at. I see a prominent amount of light on the ceiling all the way to the back of the car and right on his face. It's really straight forward to collect the actual scene luminances from this particular car interior and exterior in this location, but my estimation is the interior luminance is a bigger problem for adaptation than the street lights because the display he's presumably looking at has a much wider field of view, and he's looking directly at it for a prolonged period of time. It's possible he's not even scotopically adapted because of this.
And also why is he even looking at the screen? He's obviously distracted by something. Is this required for testing? Ostensibly he's supposed to drive the car first. Is this display standard equipment? Or is it unique to it being an Uber? Or is it an entertainment device?
Retest with an OEM lit interior whose driver is paying attention. We already know the autonomous setup failed. But barriers are in place than also increase the potential for the human backup driver to fail.
What the eye IS doing is some kind of HDR processing, which is much better than the gamma and levels applied to that video. I bet a professional colorist could grade that footage to make it a much better reflection of what the driver could see in the shadows - even with a crappy camera, you can usually pull out quite a bit of shadow detail.
Even LIDAR aside, computer vision and a raw video feed should have been enough to have prevented this collision.
When a digital camera records an image, a gamma curve is applied to it before display, which makes up for our bias against the darker portions which the digital equipment does not have. We are very capable of guessing the results of bright conditions but not dark conditions via compressed video.
Moreso, these cars should not be using consumer CCDs with compression. They should be utilizing the full possible scope of video.
Gamma correction makes up for a bias against darker portions in the display, not in our eyes. It's a holdover from the CRT days where the change in brightness between pixel values of, say, 10 and 11, was far less than the change between 250 and 251. Human eyes have excellent low-light discernment which is why 'black' doesn't really look black and you can make out blocky shapes during dark scenes on some DVDs.
Your assertion about the origins, however, are at odds with what I have been taught, my understanding, and all the supporting info I am finding in a quick search. My understanding is that luminance values from a sensor have something of an empirical scale but I’m sure this no complete explanation. I am speaking from my working knowledge. I can’t find anything supporting that it is simply a fix for discrepancies between display types. Can you link to something or explain what I am missing?
You will frequently see dash cam footage and night photography blow out the relative highlights and blacken the relative shadows.
This is because (cheap) hardware does not have the same dynamic range as human eyes, especially at night. So "properly exposed" it has to make a call to capture light values in the middle somewhere. Those light values too far out the top it interpreted as white, those out the bottom it interpreted as black, created an artificial high contrast version of what a human eye would see.
This is pretty intuitive, generally when we're driving down the road with our lights on, we aren't literally moving between pools of black, often in many urban areas I'll even forget to turn my lights on because I can see well enough.
You MAY be able to get a VERY BAD interpretation post processing of what a human would see by increasing the brightness of those pixels near the black threshold.
Also - the car is driving way too fast.
I did some driving tonight and paid close attention to when I naturally slowed down - and albeit I'm probably on the higher curve of good drivers in that I don't tailgate, drive the speed limit and generally slow much slower than the speed limit when conditions are poor (fog/rain/snow, night, slick/wet roads, near curves/hills where I can't see the road). I noticed that many of the times I naturally slowed down on the roads here I slowed considerably under the speed limit by 10 to 20 MPH in some areas. It seems this Uber SDV is generally going as fast as it is possibly allowed to regardless of what it can see.
So either way the software failed:
-If AI misjudged Lidar information and didn’t compute the slow moving pedestrian it’s a fail
-If it didn’t have enough computer vision space it should have slowed down
Possibly in the second scenario the human test driver is at fault too because he should have noticed bad condition and hit the autopilot kill switch.
With those, the driver would've seen her from a mile away.
Like a camera, your eye also has only so much dynamic range. So if those street lights are bright enough, or your interior lights are too bright, you might have nearly zero visibility in those shadows.
But it is certain that a self driving car "should" be able to see. Even two cheap digital cameras one tuned to see the darker range and the other brighter should easily see in these type situations.
Sounds a lot like rods and cones in our eyes, huh?
There's another difference with eyeballs that would almost certainly have helped here - the low light sensitive peripheral vision that the rods provide is also attuned to movement, we're more sensitive to movement in peripheral vision as well as being better able to see in low light.
Eyeballs are pretty good at night vision once adjusted, but good high sensitivity cameras can be much better. And let's not get started on LIDAR/RADAR... it seems clear to me that this was not a sensory deficiency, it was poorly designed/tested software.
I'm also very (sadly) surprised that she crossed that kind of road at night without hurrying or reacting to the sound of cars approaching.
The sign directs people to use a crossing, which is some 100 meters away at the lights around the bend.
The only thing that I think was the cars fault was that the car is programmed to drive when the driver is driving around distracted. There is no point to a human driver sitting behind the wheel of an autonomous vehicle if they aren't paying attention.
People need to understand that self driving mode isn't a freedom from the responsibility of driving safely. Rather its a tool to help ensure that driving statistically becomes safer as more self driving vehicles find their ways onto the road.
Hopefully someday all cars will be self driving and dangerous hazards/traffic reduced to the point that they are virtually none existent rather than being towards the top of the list of "preventable death" and "things humans don't want to waste most of their time during the day doing".
That's not at all clear to me. I don't know too much about cameras, but it looks to me like the camera is making the scene appear much darker than it actually is.
In the video, you can see many street lights projecting down onto the ground, and the person was walking the the gap between two streetlights. The gap between street lights (and hence the person) was in the field of view of the camera the entire time; they just weren't "visible" in the camera because of the low lighting. I'm confident my eyes are good enough that I would have been able to see this person at night in these lighting conditions. (Whether I could have reacted in time is another question.) It seems to me like the camera just doesn't have the dynamic range needed for driving in these low light conditions, which is a major problem.