I think it's not a good rebuttal because self-driving cars will have to be orders of magnitude safer than humans if order to be successful. If I'm drunk and I decide to drive and I end up crashing into a pedestrian, then it's my own damn fault and I'll face the consequences. If a self-driving car bugs out and kills someone, who's responsible?
All it'll take in a few dozen stories of self-driving cars diving into oncoming traffic like in the parent's story and that'll destroy any trust in the tech for the foreseeable future.
We're putting a driver on the road who can still learn after suffering a fatal accident! This is huge, it means all parts of the system can learn from anything and will practically eliminate deaths in the long term; much like in the airline industry. If the short term is safer but confusing we should do that and who cares about philosophical technicalities.
This attitude could literally get people killed for no measurable gain. Saving lives and reducing road deaths (which comes with a whole host of other benefits, like freeing up an enormous number of hours of people's time) is much more important than worrying about holding people responsible. We shouldn't be trying to hold people responsible for tragedies, we should be trying to prevent tragedies!
And we need to keep getting self driving cars into common use to push them up the technological learning curve . This is much more important than worrying about who is at fault because it will make the cars an order of magnitude safer. The payoffs here are so huge it is even worth putting them on the road if they are slightly less safe than humans then subsidising the producer's legal costs to hold their risk down.
The whole argument above is built upon the assumption that everybody agrees with a utilitarian mindset so I’m assuming the answer to the question, in part, is you should care about the philosophical underpinnings.
Not preventing them from doing something in the future, mind you. Holding them accountable won't be a net long term improvement. Just accountability for the sake of punishment.
I'm sure there are people who believe that, but I suspect most of the reputable philosophers will agree with the utilitarians on this particular issue.
It is difficult to defend against "your stubbornness will cause mounds of corpses for no obvious gain". This is a pretty clear cut line of approach.
"Old Uncle Bob forgot his glasses again and ran over a kid" doesn't have any implication regarding my use of a car. "A self-driving Tesla was confused by work signage and ran over a kid" would destroy a lot of trust into self-driving cars.
(I'm not a philosopher so maybe somebody with an actual background can correct me if I'm wrong)
Ignoring the hyperbole in your statement, I think the traditional counterpoint to consequentialism is deontology. Meaning, the intent behind an action matters despite the consequences. So if you are willingly putting people at risk in the short-term for the "long term good", it's morally wrong. History is rife with examples of how taking an "ends justify the means" philosophy can lead to truly abhorrent policy decisions.
>but I suspect most of the reputable philosophers will agree with the utilitarians on this particular issue.
This comes across as a "no true Scotsman" defense by implying if one is not a utilitarian in this case, they must not be reputable. Kant is generally considered a "reputable" philosopher and would likely fall on the deontological side of the argument as would many others.
>This is a pretty clear cut line of approach.
Ethics is not mathematics so there is often no "clear cut" answer. If you are defining policy, absolutism in this realm is dangerous; you have to acknowledge other people's morals may differ from yours and not fall prey to the presumption that just because it makes sense to you, it must be right.
People by and large don't intend to put people at risk. The intent of someone who causes a car crash is rarely malicious.
> This comes across as a "no true Scotsman" defense...
I suspect Kant would agree with me that it would be a difficult argument to carry as a philosopher when you are arguing in favour of increasing the corpse count for no particular reason. "If a self-driving car bugs out and kills someone, who's responsible?" certainly isn't a reason to kill people. We can afford to live in ignorance if it means more people live to die in their old age.
I'm on board with the idea that maybe someone has a reason that it is a good idea to just let people kill each other by accident. But so far in this thread nobody has bought one up.
And yet it happens, which is to the point and (depending on locality) people are still held accountable. Adopting nascent technologies almost always comes with risk tradeoffs because we don't yet have a solid understanding of the latent risks. Look at aircraft statistics if you need an example. To your point, over time they will improve, but not without accepting additional risk in the near-term. The last point is false only if you believe we understand the risk profile enough to unequivocally say autonomous cars are safer in all conditions. I don't think we can claim that yet.
What you're advocating is like someone in the early 1950s being told "Just make everyone fly in a de Havilland Comet! Don't you know air travel is going to be the safest mode of travel per mile!?" While true over the long run and in the aggregate, that individual on the Comet is being told to take an outsized risk. I'm making the claim that some people (not even necessarily me) would have a moral issue with being told at the individual level they need to take a larger risk for the betterment of society in the future.
So even while in the long time horizon they may be safer, or even be safer in the aggregate now, you could be subjecting passengers to higher risks in those individual cases. I can't speak for Kant, but I think some people may think it's immoral to draconianly force them to accept that risk. Kant speaks against the hypothetical imperative which is a motivation to obtain certain ends (like forcing policy that reduces traffic deaths in the aggregate).
If you disagree, ask yourself or others a hypothetical: take whomever you love the most in this world whether it's a child, a parent, or a dog. Would you be willing to adopt a rule today that they can never step foot in an ape-driven car for the rest of their life and every instance of travel be relegated to autonomous vehicles? Even if you are okay with that risk, there are plenty of people who would balk at it.
A driver needs to identify lanes, understand road markings and signs, model the physics of moving objects and predict the immediate future behaviour of other objects based on immediate past behaviour.
Also basing an evaluation of FSD on current AP behaviour is like assuming a housekeeper will be as effective as a Roomba.
This is false: "To ensure our statistics are conservative, we count any crash in which Autopilot was deactivated within 5 seconds before a crash."
https://www.tesla.com/VehicleSafetyReport
It would be more helpful to focus on actually deceptive issues in how they report data, namely the conflation of easy highway miles (Autopilot enabled) with more dangerous city driving (Autopilot unavailable). For example their Q1 2021 report:
"In the 1st quarter, we registered one accident for every 4.19 million miles driven in which drivers had Autopilot engaged. For those driving without Autopilot but with our active safety features, we registered one accident for every 2.05 million miles driven. For those driving without Autopilot and without our active safety features, we registered one accident for every 978 thousand miles driven. By comparison, NHTSA’s most recent data shows that in the United States there is an automobile crash every 484,000 miles."
All the self driving companies do this, but it only makes sense to compare # of accidents/mile between like processes. It's an ok proxy for total risk iff accidents were uniformly distributed, but we know that isn't even close to true.
With a system like Teslas', it is preferentially, probably approximately exclusively if normalized by distance, deployed in situations with a lower accident rate - comparing that to total rates is nonsense.
What would be more interesting is Tesla vs. human accident rates on only freeway miles, say, but I haven't seen that anywhere, have you?
Best rough stats I can find is that less than 1/2 of all accidents happen on highways. There are a also a ton of fender-bender, parking lot and/or traffic jam accidents that aren't really applicable either.
yeah maybe i was exaggerating but autopilot can definitely get you in positions where its impossible to get out of and the deactivate 5 secs before actual collision.
And autopilot is only engaged on highways but they are comparing its safety to accidents per mile in general. But those stats are strongly skewed by city and non-highway driving, where autopilot doesnt work at all.
I don't think we quite need AGI to make something like a commercial urban taxi service work. For any given place, we can substitute a very large amount of training data. And I think we can get there through an approach like Waymo, where the taxi's number one job is to ask for central human help if it's in the slightest confused or at risk of causing harm. But even that will be a very long slog.
And for the supposed goal of "full self-driving", I agree that is basically AGI, because it requires levels of judgment that even humans struggle with. The other day I came across some Covid-related road changes and I had to reason about human intent as expressed in roads while exchanging human social signals with the people around me.
Brooks suggests we instead may see people trying to simplify the problem by banning humans and/or human drivers from areas where we want automated driving. That's anthema to me, but given the history of building to favor cars vs other modes and given the vast amounts of money behind these efforts, it wouldn't shock me that it turns out that way.
I agree that ultimately they need a significant amount of local training data. But I believe they would be better off having a human drive a car around to map things out, run their 'AI' on the results, and then manually review/correct lower confidence results. Or, better yet, make a database and have cities provide feeds of changes/permits/closures.
"Sorry, I know you crashed, your car's on fire, and you're bleeding out. But I need to get this road closure form submitted before I can help, or a self driving car might ignore the stationary accident and make things worse. Ugh, 5G's not working today... hold on..."
The cars must be able to handle random, unexpected events happening without maps, and without external guidance. A wheel falls off a truck in front of you for metal fatigue reasons and is bounding down the road - that's a thing that happens, and a thing that in no way will be reported before it becomes relevant.
If a self driving car can't handle this sort of thing (or an accident and coming on a flagger) autonomously, it has no business being allowed on the road.
And if it can - then there's no need to have a centralized database of weird things, because it can deal with the range of events autonomously.
> And if it can - then there's no need to have a centralized database of weird things, because it can deal with the range of events autonomously.
'Conditions on top of the road' (vehicles, debris, etc.) and 'conditions of the road itself' are not the same thing and I'm not sure why you think that they are.
Start talking about 99.998% or so, OK, now it's worth sitting down with the math. But 98% accuracy at avoiding construction/existing crashes/etc is horrible.
Define -
1. Several.
2. Not working.
Disc: Googler but I don't know if that's necessary since I don't work at Waymo.
Self-driving will arrive eventually, but don't hold your breath.
I don’t think that’s true - but it certainly needs more than a couple of poor resolution cameras and some underpowered hardware powered by hype. You can see how poor the vision of these systems is by just watching a few Tesla autopilot videos and seeing cars flash into/out of recognition on the screen.
The first actual self driving car won’t be the one that does it with bargain basement hardware prioritising low cost over efficacy and safety.
How would your imagined autopilot system handle traffic flaggers, construction workers with signs, or various people directing traffic with handsignals?
How will your system see the difference between a police officer making hand signs that you have to obey, and some random person making hand signals?
How will it solve turn taking at stop signals, and how will it solve people waving you on, thereby changing the turn-taking order?
There are tons of driving scenarios that require you to look at other humans, figure out who they are, and figure out what they want. For humans, this is dead simple. For computers... Nope.
Any situation with hand-waving should be passed back to a human driver until the hand-waving can be eliminated with future laws and road rules. We aren't going to have full level-5 autonomy that's reliant on cameras interpreting people waggling their hands around.
The solution is that you architect a full system to allow for self driving, which will inevitably include changing how people interact with cars.
...which reinforces the point me and the grandparent post was making: You can't solve self-driving without AGI.
I guess there is a sliding scale with self driving cars:
* Low technology, high societal or infrastructure change (for example we could build a totally different road network and dig cables into it, not practical but possible with technology 20 years ago!) * Medium technology, medium societal or infrastructure change. * High technology, low societal or infrastructure change. * Very high technology, no/little societal or infrastructure change (eg AGI that can interpret hand signals).
All of these are possible versions of self driving cars - there is no “one” solution. Having AGI and going for the last bullet point isn’t the only option.
As an example, when the 1950s imagined robotic vacuum cleaning they pictured a standing robot that would use your existing appliances - so they might have said “we need to work out how to pick up and use a household vaccum cleaner in order to make a useful household robot, which requires AGI!”. We now know that a better solution is the roomba, and AGI isn’t required.
I personally think we will inevitably need some level of societal change, legislation change and possibly even infrastructure change to make self driving cars happen sooner.
[1] https://www.cnet.com/roadshow/news/waymo-self-driving-cars-p...
Also, it doesn't account for the fact that police officers can make hand signals from inside their vehicle (with or without yellow vests) that you have to obey as a driver.
Autopilot is not FSD. Autopilot is using the old system, FSD is using an architectural rewrite.
People who were not convinced by autopilot shoving cars under trucks are not going to get convinced by one more anecdote.
I think it will happen but only by making many expensive, intrusive changes to our cities and roads to accommodate "self-driving" cars.
When you look at nature and see a tiny bee or fly navigating perfectly around all sorts of obstacles, including the newspaper I’m trying to swat it with, using just that teeny tiny brain running of mili-watts of power with just compound eyes to guide it. And in 3D space not a nice flat surface.
But we still can’t make a car not drive into a massive concrete barrier?
Something is off. But that gives me hope that there’s a discovery waiting to be made that can fix it.
Yes. The fundamental assumptions about how brains work are wrong.
I see, far too often, the dismissive interpretation of brains as "A couple crappy cameras tied to a neural network" - which leads to the "Well, we have better cameras and we know how to do neural networks, so it can't be hard!" Throw in some dismissive statements about how awful humans are at driving and such, and you're set for the trap.
Human/animal vision systems are way more optimized than that sort of handwaving dismissal. The "cameras" are well optimized for what's needed, and have some pretty darned impressive scanning features to make a very small central cone of resolution do everything that's needed - without flooding the brain with HD video in places that don't matter. It's an optimized system for making a fairly narrow feed cover everything that's needed.
https://en.wikipedia.org/wiki/Saccade
And, yes, there are some weird quirks of vision systems - optical illusions and such. But once you get past the eyes, the rest of the "Make sense of the 3D world around us" system is insanely optimized, and exceedingly power efficient. It works well enough and reliably enough to handle absolutely absurd changes in the operating environments. Humans did not come born with "supersonic low level flight" built into the system, yet... our brains can work with it. It just takes some learning and brain adjustments.
And the whole rest of the system following is similarly impressive.
There's a typically paired arrogance you also see: "We are like gods in the synthetic world of the internet, because we know code and manage 99.995% API success rates, therefore we can use code to solve reality." Reality is infinitely more creative at throwing wrenches into things than we are at solving them with code, and while it doesn't really matter on the internet, it very much does matter at 75mph.
When you dismiss all that as "crappy cameras and a neural network, we know code, we can do that!" - you end up failing in the predictable ways we see with the self driving cars.
The power gap is so huge, our software simply must be inadequate. We're talking about a power gap wider than the gap between my toaster oven and the Saturn V.
I just found this weird video of how a dragon flies brains act basically like a homing missile, using only a few dozen neurones: https://www.youtube.com/watch?v=Y0vRupFPw90
If driverless cars were crashing because all of a sudden a giant sheet of glass appeared in the middle of the road, or a giant alien death ray shot up the tarmac, I think most people would have some sympathy.
These are entirely avoidable problems using the sensors and methods every other company besides Tesla are using
Sensors might be an alternate strategy though and a way to at least take vision processing out of the list of hard problems.
If we used genetic algorithms to design the algorithm, it'd probably be able to get something as efficient as that. However, you're also not going to be able to "fix this one thing" because it's more likely to be the result of emergent behavior from interactions more complicated than anyone can understand. I believe one could get something that works like that, but you're also going to have other behaviors akin to moths flying into lamps because they navigate by the moon which is recognized as "the brightest thing at night".
We think we can make rules for everything and expect them to be followed without exception to perform these complicated tasks, but I think that's hubris. Anything that ends up doing it at that power scale is going to be as inscrutable as a brain. Asimov's robopsychology is then likely to be something real, but it's going to be dissecting neural networks because "fix this issue" on something that took multiple GW of energy to train in the first place will need fixing from that end versus "well, time to train it again".
As an anecdote for this thread, I remember there was some setup where physical circuits were designed via genetic algorithms to do some task. There were results with unconnected resistors that actually influenced the behavior of the circuit. You're generating behavior which is sensitive to that level. Wish I could find that again…
We could put some metal guides in the ground so they can follow that path.
We could then separate that path from other obstructions with barriers, maybe put some tunnels in the ground that keep other people out.
And have signals that indicate when the "self driving" stuff is going to interfere with other roads, so nobody gets in the way!
"Totally change cities to meet the requirements of self driving cars" sounds an awful lot like "Just put rails in and call it a train."
In many countries it would be a huge culture shift, and IMO a step in the wrong direction. We don't need more urban real-estate devoted to cars.
NY to Jersey - nope.
In the rich world it is not hard to find a potholed road with faded markings, so how can we expect our society to maintain and upkeep expensive infrastructure when we can barely manage paint and basic asphalt?
The capability of the system would increase as more nodes are added. Kind of like the Geth from the Mass Effect series.
The idea the full AGI is required is silly and incorrect. Waymo does not have these issues at all. It couldn't happen either. You don't need AGI to have a detailed map, you don't need AGI to properly localize to a detailed map.
these are problems Tesla is giving itself
False:
https://jalopnik.com/watch-a-waymo-av-get-freaked-out-by-tra...
that is not related to the tesla issue at all. Furthermore, this article just described bad decision making with regards to traffic cones, however was not dangerous. It was aware of drivable surfaces and direction of traffic... because it has a map
Do you see how these are different situations?
1. Telsa swerving into an oncoming lane 2. Waymo coming to a controlled stop in front of a traffic cone, then thinks it is blocked and needs help
And then driving off again, and then blocking traffic, and then evading Waymo's roadside assistance, etc.
Do you see how these aren't different situations?
That is fundamentally different than trusting a neural net to tell you where the lanes are, where the objects are, and what kind of objects they are.
you do not drive into an oncoming lane if you have a hd map. you do not drive into a concrete wall if you have lidar.
Not having control of the car is high danger. Even when they recognized it needed rescuing they still didn't have control of it.
How do you define full AGI?
On a side note, another approach would be to have sensors in the roads that would transmit boundary information. That might be too expensive and difficult to maintain though.
In fact, I believe that the difficulty of even _defining_ AGI is yet more evidence that we are not anywhere close to it.
What I think an AGI needs to have is the _ability to learn_. To me, that's what intelligence really is. For many people, it's having a lot of knowledge. However, that is merely lack of ignorance.
The reason that the ability to learn is important for AGI is because the true thing we miss with self-driving cars is _adaptability_. When roads change, humans adapt. But a self-driving car does not, unless a human interferes and tells it, "You learned about cones today."
FWIW, I'm not okay riding in a car that's driven by ~50% of nonprofessional human drivers; people are, in general, really bad at the task.
The old system. Current Autopilot is using the old system. Not relevant to the new FSD beta.
BTW, the average driver gets in a crash every 18 years. The best humans aren't perfect either, and a self-driving car really only needs to be better than the worst humans to be an improvement (although it will have to be much better to succeed).
I think this is false because varying abilities of people will use the autopilot, not just the worst. The autopilot needs to be better than average for an overall improvement. I think it needs to be much better than average to be compelling enough for most people to adopt it. Most people consider themselves to be good drivers, even with sufficient evidence to the contrary.
That depends entirely on how effective it is and what incentives/disincentives there are (i.e. subsidies, taxes, insurance), which will itself depend on how effective it is.
No one ever said that they could get all of their info from a database. But there is a significant amount of info which they could get from a database, which they might not be able to get at all otherwise.
Car A (driven in manual mode) tells Car B (FSD) that tells Car C (FSD) that tells... Car Z (FSD): "I stopped at a stop sign in stop & go traffic at Lat/Long xyz while I spotted a human in the middle of the road. This is marked on my maps data (updated overnight) as not being a normal stop. No new data was detected except the human standing and a human laying on the ground."
Car B, C, D... all confirm the same until Z passes the area and sees no more humans (they've been moved). Car Z tells Car AA all good here, proceed as normally.
I'm just rambling now, but it's truly how ants operate. Draw a line of chalk in the middle of their "line" or walking patch and watched them avoid it and find a way around it. The difference is all of these cars will feed a true neural net of public data of live road conditions, accidents, etc. A possible privacy nightmare, but these are different times.
In your case, the new lane was clearly marked, which just shows how weak the system really is. Imagine if there was snow covering the lane markings. Or sunlight reflecting off of the road confuses the vision.
People bought the self-driving hype because the first 80% was easy. They forgot that the next 20%, as usual, would be incredibly hard, if not impossible.
Personally, I will never be okay riding in a car that is on autopilot. Heck, I'm not even okay with the amount of automation in airplane cockpits. [1] [2]
[1]: https://www.nbcbayarea.com/news/local/faa-report-pilots-addi... [2]: https://hbr.org/2017/09/the-tragic-crash-of-flight-af447-sho...