News for hackers. Stuff that matters.
Re:Another Tesla on autopilot steers towards a barrier by jacquesm on Friday April 06, 2018 @10:39AM (#16773551)
I totally agree with your analysis. What is more worrying is that even following the rest of the traffic would have solved the problem. If you are the only car doing something then there is a good chance you are doing something wrong.

Also: the lane markings are confusing but GPS and inertial readings should have clearly indicated that that is not a lane. If two systems give conflicting data some kind of alarm should be given to the driver.

Re:Another Tesla on autopilot steers towards a barrier by lisper on Friday April 06, 2018 @11:00AM (#16773731)
> lane markings are confusing but GPS and inertial readings should have clearly indicated that that is not a lane

GPS is not accurate enough to reliably pinpoint your location to a particular lane. Even WAAS (https://en.wikipedia.org/wiki/Wide_Area_Augmentation_System) can have up to 25 feet of error. Basic GPS is less accurate than that.

In fact, it's possible that GPS error was a contributing factor here but there's no way to know that from the video.

Re:Another Tesla on autopilot steers towards a barrier by ac29 on Friday April 06, 2018 @12:42PM (#16774660)
RTK [0] systems definitely can have centimetre-level accuracy, though they require a fixed base station within 10-15 miles or so to broadcast corrections. It would also require roads to be mapped to a high level of accuracy.

I have seen self-driving test cars in Silicon Valley (frequently, especially in the last year or so) using these types of systems, so they are at least being tested. I've also heard discussion of putting RTK base stations on cell-phone towers to provide wide area coverage, but I'm not sure if much effort has been put into that. I do know vast areas of the agricultural midwest are covered in RTK networks -- its used heavily for auto-steering control in agriculture.

[0] https://en.wikipedia.org/wiki/Real_Time_Kinematic

Re:Another Tesla on autopilot steers towards a barrier by fma on Friday April 06, 2018 @01:54PM (#16775520)
I've heard on and off for many years (15 years?) about ad-hoc wireless networks for cars. Whether it's car to car, or car to some devices on the ground (lane marking, stop lights) so it knows where it is at in relationship to the road and other cars - and it would know if the car ahead is going to break or slow down.

Now the cars are relying on cameras, lidar to figure things out. What has happened to putting sensors on the road to broadcast what/where the roads is. Is that out of the question now because of cost?

Re:Another Tesla on autopilot steers towards a barrier by jacquesm on Friday April 06, 2018 @01:59PM (#16775578)
I can see lots of potential problems with that, for instance you'd have to rely on the sensor data of that other car and you'd have to believe that other car telling you the truth. Lots of griefer potential there.
Re:Another Tesla on autopilot steers towards a barrier by jacquesm on Friday April 06, 2018 @11:05AM (#16773773)
I drive a lot with my GPS (TomTom) on and it rarely gets the lane I'm in wrong, usually that only happens in construction zones.

So even if the error can be large in practice it often works very well.

It would be very interesting to see the input data streams these self driving systems rely on in case of errors, and in the case of accidents there might even be a legal reason to obtain them (for liability purposes).

Re:Another Tesla on autopilot steers towards a barrier by lisper on Friday April 06, 2018 @11:17AM (#16773863)
> it often works very well

Well, yeah, it's not like autopilot is driving cars into barriers every day. But I don't think "often works very well" (and then it kills you) is good enough for most people. It's certainly not good enough for me.

Re:Another Tesla on autopilot steers towards a barrier by danso on Friday April 06, 2018 @11:38AM (#16774045)
According to the Reddit posterL

> Previous autopilot versions didn't do this. 10.4 and 12 do this 90% of the time. I didn't post it until I got .12 and it did it a few times in case it was a bug in 10.4.

> Also, start paying attention as you drive. Centering yourself in the lane is not always a good idea. You'll run into parked cars all over, and lots of times before there is an obstacle the lane gets wider. In general, my feel is that you're better off to hug one of the markings.

It kind of does sound like that autopilot is steering vehicles into barriers everyday, or it would be if drivers weren't being extra vigilant: https://www.reddit.com/r/teslamotors/comments/8a0jfh/autopil...

Seems like we need a study to assess whether the kind of protections AP offers outweighs the extra challenges it adds to a daily drive.

Re:Another Tesla on autopilot steers towards a barrier by jacquesm on Friday April 06, 2018 @11:22AM (#16773913)
That's absolutely true, however, I did not mean it to take the place of whatever they have running right now, I meant for it to be an input into a system that can detect if there is a potential problem and the driver should be alerted.

Because if I interpret the video correctly either the car was partially in the wrong lane before the near accident or it was about to steer it into a situation like that. And that means it is time to hand off to the driver because the autopilot has either already made an error or is about to make one.

Either way the autopilot should have realized the situation is beyond it's abilities, and when GPS and camera inputs conflict (which I assume was the case) that's an excellent moment for a hand-off.

Re:Another Tesla on autopilot steers towards a barrier by aequitas on Friday April 06, 2018 @11:09AM (#16773810)
Don't navigation systems 'snap' to the most probable lane you are driving in based on general direction instead of your exact GPS coordinates?

A good example of this I think is when driving in tunnels, there is no GPS info there, but the navigation shows you following the (curved) road.

Re:Another Tesla on autopilot steers towards a barrier by ryandrake on Friday April 06, 2018 @11:22AM (#16773914)
Good nav systems will fuse whatever inputs they can get including GPS, INS, compass, etc. in order to figure out and ‘snap’ to what road you’re on. Snapping to an individual lane is still AFAIK beyond these systems’ sensors’ capability. You will need an additional input like a camera for this that looks for lane markings.
Re:Another Tesla on autopilot steers towards a barrier by jacquesm on Friday April 06, 2018 @11:12AM (#16773829)
Yes, they do that, and they rarely get that wrong.

Also, the better ones have inertial backup & flux gate for when GPS is unavailable. Inertial backup could also help to detect the signatures of lane changes and turns.

Re:Another Tesla on autopilot steers towards a barrier by saagarjha on Friday April 06, 2018 @04:32PM (#16777128)
Try driving near a place with a lot of roads that are close together, like an airport. You’ll quickly see that snapping doesn’t really work, especially if you deviate from the route the GPS is trying to take you on.
Re:Another Tesla on autopilot steers towards a barrier by jacquesm on Friday April 06, 2018 @05:06PM (#16777439)
I do this all the time and it works remarkably well. Amsterdam and the roads around it are pretty dense, up to 6 lanes each way in quite a few places with all kinds of flyovers and multi-lane exits.
Re:Another Tesla on autopilot steers towards a barrier by galdosdi on Saturday April 07, 2018 @10:07AM (#16780752)
At what speeds? I'm in NYC, similar dense road systems. At a reasonable speed, snapping still works because it knows where you came from. In stop stop stop go traffic combined with lane weaving and ramp merging I've found it is not hard to trick the gps into thinking you're on a parallel access road or unrelated ramp for a little while.

It's possible your highways are newer or less congested than ours?

Re:Another Tesla on autopilot steers towards a barrier by ams6110 on Friday April 06, 2018 @11:35AM (#16774015)
Counterpoint, my Lenovo tablet running google maps often seems to get confused about lanes. E.g. it will think I'm taking an exit ramp when I am still on the main road. Once the ramp and the road diverge enough it will recover, but not before going into it's "rerouting" mode and sometimes even starting directions to get back on the road I'm already on.
Re:Another Tesla on autopilot steers towards a barrier by jacquesm on Friday April 06, 2018 @11:46AM (#16774127)
That sounds pretty bad if that's the thing you rely on to get you where you're going. Here in NL interchanges are often complex enough that without the navigator picking the right lane you'd end up taking the wrong exit.

I don't use Google maps for driving but I know plenty of people that do and having been in the car with them on occasion has made me even happier with my dedicated navigation system. The voice prompts are super confusing with a lot of irrelevant information (and usually totally mis-pronounced) and now you are supplying even more data points that the actual navigation itself isn't as good as it could be.

I suspect this is a by-product of doing a lot of stuff rather than just doing one thing and doing that well.

Re:Another Tesla on autopilot steers towards a barrier by rayiner on Friday April 06, 2018 @12:01PM (#16774262)
Here in D.C. Google maps will regularly get confused about whether you're on the main road or a ramp; whether you're on an overhead highway or the street underneath it, whether you're on a main road or the access road next to it.
Re:Another Tesla on autopilot steers towards a barrier by reaperducer on Friday April 06, 2018 @03:53PM (#16776793)
My FIATChrysler Uconnect system will sometimes think I'm on the road above when driving under a bridge.

Sometimes it'll shout "The speed limit is 35 miles an hour!" when I go under a bridge carrying a 35MPH road, even though the road I'm on has a limit of 65.

Re:Another Tesla on autopilot steers towards a barrier by aws_ls on Friday April 06, 2018 @11:47AM (#16774137)
Is it using some other localization technique to complement the GPS? GPS at best can give accuracy within 3 meters (if I remember right). To make it accurate, laser/radar beams are shot out to the landmarks, of which precise co-ordinates are known, and using them the pin point coordinates of Ego vehicle are determined.
Re:Another Tesla on autopilot steers towards a barrier by michaelt on Friday April 06, 2018 @03:17PM (#16776483)
The highest resolution GPS receivers have two changes:

1. Base stations at known locations broadcast corrections for factors like the effects of the ionosphere and troposphere, and inaccurate satellite ephemerides. If you have multiple base stations, you can interpolate between them (Trimble VRS Now [1] does this, for example)

2. Precise measurements by combining code phase (~300m wavelength) and carrier phase on two frequencies (~20cm wavelength), plus the beat frequency of the two frequencies (~80cm)

With these combined, and good visibility of the sky, centimetre-level accuracy is possible.

Autonomous vehicles will often combine this with inertial measurement [2] which offers a faster data rate, and works in tunnels.

Many people also expect autonomous vehicles to also track lane markings, in combination with a detailed map to say whether lane markings are accurate.

[1] http://www.trimble.com/positioning-services/vrs-now [2] http://www.oxts.com/products/rt3000/

Re:Another Tesla on autopilot steers towards a barrier by jacquesm on Friday April 06, 2018 @11:53AM (#16774202)
Inertial, flux gate compass.
Re:Another Tesla on autopilot steers towards a barrier by lightedman on Friday April 06, 2018 @12:23PM (#16774447)
"GPS is not accurate enough to reliably pinpoint your location to a particular lane."

My phone has GPS accurate to within 1 foot. I use it for mining location all the time. It uses differential GPS plus Inertial sensors.

Re:Another Tesla on autopilot steers towards a barrier by rayiner on Friday April 06, 2018 @10:52AM (#16773648)
When I think of self-driving car technology, I think of all of these inputs (maps, cameras, GPS, etc.) being fed into a world model from which inferences are drawn.[1] Here, the car could have known from cameras and maps that there was a lane split, and should have been able to detect that the line following was giving a conflicting inference compared to those other sources. The question is: does Tesla actually do that? Because if not, it's not really self-driving car technology. It's not a glitch in a product that's 90% of the way there; it's a fundamentally more basic technology.

[1] See https://www.ri.cmu.edu/pub_files/2009/6/aimag2009_urmson.pdf at 21-23.

Re:Another Tesla on autopilot steers towards a barrier by jacquesm on Friday April 06, 2018 @11:19AM (#16773879)
The whole problem seems to me that to get a 90% solution for a self driving vehicle is now no longer hard. The remaining 10% is super hard and the degree to which the present day solutions tend to get these wrong is worrisome enough that if the quality doesn't jump up very quickly the whole self driving car thing could end up in an AI winter all of its own making. And that would be a real loss.

This is not a field where 'move fast and break stuff' is a mantra worth anything at all, the stuff that is broken are people's lives and as such you'd expect a far more conservative attitude, akin to what we see in Aerospace and medical equipment software development practices.

But instead barely tested software gets released via over-the-air updates and it's anybody's guess what you'll be running the next time you turn the key.

I agree with you that the software should have been able to detect that something was wrong either way, either it was already halfway in the wrong lane or it was heading to be halfway in the wrong lane, a situation that should not have passed without the car at least alerting the driver.

And from what we have seen in the video it just gave one inferred situation priority over the rest and that particular one happened to be dead wrong about it's interpretation of the model.

Re:Another Tesla on autopilot steers towards a barrier by ansible on Friday April 06, 2018 @03:31PM (#16776596)
I agree with all you've said.

One of my concerns with self-driving systems is that they don't have a good model of the world they are operating in. Yes, sure, they build up 3-D models using multiple sensor types, and react to that accordingly, oftentimes with faster reflexes than a human.

However, consider this situation with pedestrians A and B, while driving down a street, approaching an intersection.

Pedestrian A, on the edge of the sidewalk, is faced away from the street, looking at his phone. The chances of him suddenly stepping out in front of my car are exactly 0.0%.

Pedestrian B, on the edge of the sidewalk, is leaning slightly towards the street, and is looking anxiously down the street (the wrong way). The chances of B suddenly stepping out is high, and the chances of B not seeing my car are very high (because B is being stupid/careless, or else is used to UK traffic patterns).

I, as a human, will react to those situations differently, preemptively slowing down if I recognize a situation like with Pedestrian B. Autonomous driving systems will treat both situations exactly the same.

From everything I've read about so far, current driving systems have no way of distinguishing between those two specific situations, or dealing with the myriad of other similar issues that can occur on our streets.

Re:Another Tesla on autopilot steers towards a barrier by rtx on Friday April 06, 2018 @04:07PM (#16776914)
Another similar situation where autopilot may fail. It sees a ball rolling across the street. Will it slow down, because a child maybe chasing the ball.
Re:Another Tesla on autopilot steers towards a barrier by seaurchin on Friday April 06, 2018 @05:38PM (#16777674)
I used to contemplate similar situations. For example, a stationary person on a bike at the top of a steep driveway that's perpendicular to the vehicle's driving direction. But after seeing the video of the Arizona incident I stopped. It's too soon to consider these corner cases when some systems on the public road can't even slow down for a person in front of the fucking vehicle.
Re:Another Tesla on autopilot steers towards a barrier by galdosdi on Saturday April 07, 2018 @10:11AM (#16780769)
That's because waymo is playing chess while uber plays checkers. For waymo, these corner cases probably matter. Uber is not actually pursuing self driving tech, they are pretending to do so to defraud investors. Their troubles aren't representative therefore.
Re:Another Tesla on autopilot steers towards a barrier by volkl48 on Friday April 06, 2018 @05:03PM (#16777413)
The saying in those fields is "the rules are written in blood". The Therac-25 (a medical device) is taught in practically every CS Ethics class for having changed software development practices for such things.

The cynic in me thinks self-driving will require the same, the rules and conservative practices will only come after a bunch of highly publicized, avoidable failures that leave their own trail of blood.

Re:Another Tesla on autopilot steers towards a barrier by jacquesm on Friday April 06, 2018 @06:05PM (#16777893)
I wonder how the public would respond if the person killed in that Uber accident a while ago was a kid on a bicycle and how the response will be when/if a Tesla on autopilot plows into a schoolbus (a situation they seem to have problems with is stationary objects partially in a lane).
Re:Another Tesla on autopilot steers towards a barrier by Piskvorrr on Friday April 06, 2018 @06:06PM (#16777901)
If there's anything left to be conservative upon. Current climate of "drive fast and kill people" might wipe out the field via PR.
Re:Another Tesla on autopilot steers towards a barrier by TarpitCarnivore on Friday April 06, 2018 @04:04PM (#16776884)
This is where my chief complaint with Musk comes in. His whole bravado as a salesman is probably having a longer term impact on AI for cars. He keeps pushing this auto pilot when it's not.
Re:Another Tesla on autopilot steers towards a barrier by bra-ket on Friday April 06, 2018 @03:48PM (#16776748)
regardless of the lane recognition it has to at least detect the obstacle in front of it, it's something that a basic collision avoidance system can do
Re:Another Tesla on autopilot steers towards a barrier by danso on Friday April 06, 2018 @11:35AM (#16774017)
> What is more worrying is that even following the rest of the traffic would have solved the problem.

That's what makes this example bizarre to me. I had thought that AutoPilot's ideal situation was having a moving vehicle in front of it. For example, AP does not have the ability to react to traffic lights, but can kind of hack it by following the pace of the vehicle ahead of it (assuming the vehicle doesn't run a red light):

https://www.youtube.com/watch?v=EXs4qZlDWbQ

Re:Another Tesla on autopilot steers towards a barrier by Piskvorrr on Friday April 06, 2018 @06:12PM (#16777948)
That's a heck of an ass-u-mption. Running the red light as the n-th exponentially increases the probability you'll get into collision.
Re:Another Tesla on autopilot steers towards a barrier by redler on Friday April 06, 2018 @11:22AM (#16773907)
If you are the only car doing something then there is a good chance you are doing something wrong.

I’m surprised self-driving systems don’t do this (do they?). One or more vehicles in front of you that have successfully navigated an area you are now entering is a powerful data point. I’m not saying follow a car off a cliff, but one would think the behavior of followed vehicles should be somehow fused into the car’s pathfinding.

Re:Another Tesla on autopilot steers towards a barrier by jacquesm on Friday April 06, 2018 @11:24AM (#16773931)
"Follow the leader" is a main ingredient in flocking behavior in birds, who rarely if ever collide in flight.
Re:Another Tesla on autopilot steers towards a barrier by thaumasiotes on Friday April 06, 2018 @11:22PM (#16779216)
> If you are the only car doing something then there is a good chance you are doing something wrong.

Following this principle would probably result in a lot of people angry that their autopilot had gotten them a speeding ticket.

Re:Another Tesla on autopilot steers towards a barrier by bouke on Saturday April 07, 2018 @01:51AM (#16779559)
> lane markings are confusing but GPS and inertial readings should have clearly indicated that that is not a lane

Aside from GPS’ accuracy as mentioned in other replies, also take into account the navigation’s map material. The individual lans are probably not individually tracked on the map, but a single track per road with meta data specifying the number of lanes amongst other featerus. So even if GPS would have provided very accurate position readings, the map source material might not even match that level of detail.

Re:Another Tesla on autopilot steers towards a barrier by matt-attack on Friday April 06, 2018 @04:22PM (#16777036)
Pretty sure I heard an audible warning in the vid.
Re:Another Tesla on autopilot steers towards a barrier by jijojv on Saturday April 07, 2018 @02:31AM (#16779654)
No, that's sound you hear when you override and take over the autopilot