I've got some bad news for you.
1. https://www.nimh.nih.gov/health/statistics/mental-illness
It remains to be seen if autonomous driving systems are actually safe. But if the other driver does something that is safe, there's then an onus on the first driver to have accounted for that.
I don't disagree, but in order for the first driver to "account" for the actions of the second, they have to have some reasonable ability to predict what that driver will do. That gets us back to the theory-of-mind question.
I'm willing to bet that's around 10 times what you were considering as a safe following distance in your head, and probably still 5 times more than what you were picturing for the safe distance behind a brake checking AV.
As more and more cars default to this safely, however, it’ll start to equal out.
If you don't maintain a safe following distance for your speed, you are the one creating the dangerous driving environment. Tailgating is worse for both traffic flow and safety.
Tailgating is bad, regardless of if people brake check or not. If automous vehicles are what it takes to get you to stop tailgating and follow at a safe distance, then that is just an added bonus.
Why, for example, do you think trainers post "Student Driver" stickers on their cars? It's because it signals the driver may be more unpredictable and people (rightly) tend to give them wide berth. You're essentially advocating that everyone treat everyone else (and every robot) as a student driver. That's fine for a dichotomous safety mindset, but other people would prefer to recognize the tradeoffs with that approach.
Or maybe you're just deliberately bent on misunderstanding my point, I can't read your mind :)
What is this nonsense? The safe following distance is determined by how fast you can stop, not by who is driving the vehicle you are following.
> Why, for example, do you think trainers post "Student Driver" stickers on their cars? It's because it signals the driver may be more unpredictable.
No, it signals they have less experience and are more dangerous drivers. When it comes to driver predictability, student drivers tend to be far more predictable than the adult, overconfident drivers. I've never seen a student driver roaring past me in stopped traffic on a shoulder, or floor the gas to pass me through a light because they didn't want to turn in a turn only lane, or any of the other unpredictable things I see on a regular basis from experienced drivers.
> You're essentially advocating that everyone treat everyone else (and every robot) as a student driver. That's fine for a dichotomous safety mindset, but other people would prefer to recognize the tradeoffs with that approach.
I think that if more driver treated the people around them as student drivers, our roads would be a lot safer.
I know that if people followed at a safe distance then we would have fewer traffic jams.
Edit: You also seem stuck on the idea that Waymo unsafely unexpectedly brakes more often than human drivers, yet that isn't clear to me from the data we have. Indeed it seems like the opposite is true from the data.
So when you're driving, do you somehow know the braking distance of every car and reaction time of every driver around you? You don't, and since their braking distance is needed to know your own braking requirements, you have to use heuristics. Maybe your heuristic is "assume everyone will cram on the brake, full tilt, at any time." But, that is not a pragmatic solution given our current infrastructure. We don't have the road capacity for everyone to drive that way. So we make tradeoffs. Part of that tradeoff means anticipating what other drivers will do and adjusting accordingly. Naturally, this will trade some safety for other things we value. That is the reality of the world we live in. You seem to be advocating for something else. The OP was that we might struggle to apply such heuristics without a theory of mind to guide us.
We probably just disagree on the student driver vs. overconfident drivers. I feel like I'm pretty good at anticipating aggressive drivers, and I fear them must less than the super-tentative driver that tends to put other people at risk. But unless you have data, we're just talking about subjective opinion here so it's not really worth delving into further.
>I think that if more driver treated the people around them as student drivers, our roads would be a lot safer.
Sure. But again, it doesn't really fit with the world we live in. Should we all, in general, drive more defensively? Sure. But I doubt our infrastructure will allow for 25+ car lengths between vehicles that the NHTSA recommends, so we're stuck making some tradeoffs.
I agree on the data point. I'm not making strong claims about safety. I'm making claims about uncertainty. One thing that is clear (and I've advocated elsewhere) is that we don't have good data (in part, because companies get to share only what they want in many cases), which makes uncertainty greater.
25+ car lengths between vehicles that the NHTSA recommends
Note: 75 mph is 120 kmh
You can (usually) follow a large semi a bit closer because its braking distance is longer than yours.
But because of reduced visibility you can end up with “revealed brake checkmate” where the semi swerves into the next lane because a vehicle is stopped in the lane, which you then need to swerve or hit.
It’s more effective to let your initial point stand and let the discussion run its course. You’re working against yourself now.
More following distance => less throughput. There is already a math problem at play for how to get millions of single occupancy to and from a given location in a day. Which is to say, gridlock occurs today - it is a fact. Increasing following distance by 2 to 5x would potentially be quite bad for otherwise heavy but flowing traffic. Eg: NYC, it is known for good throughput. Light turns green, everyone picks up speed and keeps tight following distances. Anything less and that city would lock up more so than it already does.
Part of the issue is just complexity and variety. 2x following distances on most highways would likely be only a good thing. But in each and every circumstance, not necessarily good, perhaps disaster.
I think it can fun to be the driver that smooths out the shock waves, so I do largely agree with your point. Though, all things being equal, double following distance and that will double the length of a traffic jam. Longer traffic jams are more impactful - it takes longer to get through them & the longer traffic jam distance is more likely to cause back-ups on on-ramps and in turn to the surface streets leading to those on-ramps. Which is to say, a higher prevalence of grid-lock.
The AI crowd is right to say that with 100% AI cars, you can have smooth and fast traffic with impossibly tight following distances. I do believe you need some impractical necessities to accomplish that though.
You keep making the same argument over and over even though I've repeatedly explained that it does not match our understanding of how traffic jams form. Traffic jams are caused by braking, especially hard braking. Tailgating increases the need to brake hard if the person in front of you brakes or someone needs to merge. Following closely does not increase throughout, it is simply bad driving with no upside.
> I'm pretty good at anticipating aggressive drivers, and I fear them must less than the super-tentative driver that tends to put other people at risk.
Wow, if you read the behavior I descrived as "agressive driving" then we have completely different standards. I was describing reckless driving that blatantly violates the rules of the road.
Stop using excuses for your bad behavior and you might even be able to become a better driver.
>Traffic jams are caused by braking
Guess what one of the main issues has been in AV...nuisance braking. So much so that they suppress safety-critical actions to avoid unwarranted braking.
I also think you're confusing the proximate causes of conjestion for the root causes. Traffic conjestion is a load sharing problem. Tailgating is, in part, a symptom of inadequate capacity. You are advocating a solution that exacerbates it by reducing carrying capacity.
To reiterate (yet again) we aren't in disagreement about whether slowing down, or increasing following distance, will increase safety. It will, but also, that's not the point I was making. I'm just saying that is a superficial understanding of the problem and you aren't accounting for the tradeoffs. Those tradeoffs are the reason your proposal misses its mark.
Why are we still debating something that has been answered by science? Stop assuming your intuitions are correct and look at what the science actually says about how follow distances affect the maximum capacity of a road.
That said, if you routinely tailgate the driver in front on the assumption that nothing will go wrong then you've chosen to accept the consequences when an otter runs out in front of them (hey I didn't see that coming either) and they suddenly limit brake and you're sitting in their back seat. Or a steering tie breaks in a classic car coming the other way, which hits the car in front of you head on and now they're stationary.
The question of how much additional caution (in terms of lower speed limits, longer following distances etc.) is optimal in terms of overall QALYs is, I feel, vastly under-considered and under-discussed.
Funny story, I actually "crashed" in a hot air balloon as a kid, when the hill we were landing on had a draft running up over it that caught the balloon after the basket had touched down, and dragged us along the ground sideways for a good quarter mile.
This is half remembered from a Snowmass balloon rally conversation.
Basically if on the same level the one on the other’s right has the right of way.
But a plane in distress has right of way over the balloon.
I don’t think you can make this strong of a claim with the available data. Likewise, someone can’t make strong claim in the opposite direction. The best data I’ve seen is from NTSB investigations, and it clearly shows some very dangerous behavior. But it’s just a snapshot of data.
And I think you’ve taken away the wrong point. It was about unexpected behavior, not “driving with a death wish.”
The ability of people to consistently miss/twist the point to fit their own predetermined viewpoint is tiresome.
Indeed - so how did systems with a "death wish" enter into this discussion? Well, it turns out it was by you, about which you said "...maybe not so good on a public road."
In this light, your complaint about others twisting the point seems rather ironic.
It was the way the pilots described the unexpected behavior of the RL model.
If I related a story about ChatGPT where someone said, "It wrote like it was drunk" would you insist that I'm saying ChatGPT actually imbibes in alcohol before coming up with a response? I think you might be missing the point of an analogy. This tends to correlate with people with dichotomous thinking, which is also part of the thread and the difficulty with people understanding the intended point.
The part you missed was that the "maybe not so good on a public road" was about breaking "best practices," not about acting with a death wish. The intent was to underscore, yet again, was that unexpected behavior is sometimes beneficial in a wartime environment where you want to keep the other party guessing and off-balance, but not beneficial in a public safety domain. Again, twisting an argument to make a preconceived point rather than reading it as it was actually written. It's hard to get someone to understand a point when their biases are hell-bent on deliberately not understanding it.
And you chose to quote it, and to make the point that it is not the sort of thing we want on public roads - as if there's the slightest hint in the extensive testing of these cars that the apocalyptic scenario you chose to introduce to this discussion was anything but hypothetical and hyperbolic FUD.
And then you had the effrontery to chide dcow for responding to your claim, as if they were taking the discussion in this absurd direction. You've chosen to repeat yourself here, so I will say again that there is considerable irony in what you are doing - irony in the sense of statements being made that display a lack of self-awareness.
> And you chose to quote it, and to make the point that it is not the sort of thing we want on public roads - as if there's the slightest hint in the extensive testing of these cars that the apocalyptic scenario you chose to introduce to this discussion was anything but hypothetical and hyperbolic FUD.
At some point there were numerous reports of cars using driving assist steering into the dividers at highway offramps. I believe there were in fact some some real accidents that happened this way. That definitely was unexpected behaviour and I would argue one could even qualify it as the car acting like it had a death wish so I don't think the OP statements can be qualified as hyperbolic FUD.
Now you're going to argue that these were early incarnations of non AV systems (despite the name), but I think they do illustrate how the systems can behave in unpredictable (and dangerous) ways when they encounter novel situations. That's why I commend waymo for not following the hype and keeping the environment they operate very restricted.
Well, yes I am, because, as you realize, it is a valid point! I will also add that, I have, myself, made the point that we have to be careful because complex systems have bizarre failure modes, so just because, in everyday circumstances, these systems may appear to function 'sensibly', we cannot simply trust them to be 'sensible' outside of the scope of testing.
I am, however, a believer in the relevance of empirical evidence, and I also agree (with some caveats) that human-driver performance is a valid basis for establishing whether, and to what extent, autonomous vehicles may be permitted on public roads. We are discussing the publication of some results of Waymo's extensive testing, and I stand by what I wrote in my first response here: none of these three cases involved the Waymo car behaving in ways that are not that uncommon among human drivers, and our theory of mind does not make us nearly-infallible predictors of what another driver is going to do.
I agree with you that Waymo is taking the right approach here (and, FWIW, I regard Tesla's two-faced stance as unethical.)
I doubt I would have objected to Bumby's invocation of a death wish in response to the past events of which you speak, but in response to Waymo's test results and the points I made about them, I think 'hyperbolic FUD' is justified. At some point, a rational person has to make accommodations when things have changed and arguments based on old data lose their relevance.
This bypasses the entirety of the trust argument originally stated by leveraging the very point (statistics) it cautioned against. (As stated elsewhere: "So all the bleating about statistics may be necessary, but not sufficient, to get wide-scale adoption of AVs on public roadways.")
>none of these three cases
Regardless if you think the decision should be based on statistics alone, my further point is that an n=3 sample size is not adequate to make strong claims. Add to it that Waymo only reports the data they want to[1], there is a reason to be careful about making any claims.
Here's how that scenario plays out when we rely on organizations to self-report safety data in a competitive environment, from my experience. Even though it may start with the best of intentions, cost and schedule pressure builds. Now items that would have otherwise been reported as safety incidents are now classified with vague terms like "test anomalies" and essentially buried. They will still report safety metrics, but now it's, at best, incomplete and misleading. Until some event that's egregious enough forces the company to be more transparent.
[1] https://spectrum.ieee.org/have-selfdriving-cars-stopped-gett...
We know they aren't fully transparent. We also know that other well-known and well-funded AV developers have very bad practices that are highlighted when they are forced by regulators to be transparent. While that isn't a smoking gun against Waymo, it should give us pause and make one question a naive perspective in favor of a skeptical one.
You made two points right next to each other, about sample size and bias, and the sample size point was invalid.
When called out on that, you don't get to move the goalposts and say that the second one was "the point" and the other person is "missing the point".
Given the protracted nature of this thread, I get why it's confusing. I have a couple sub-threads that make those two points separately, and was simply trying to show how both are related. If one isn't aware of the broader context of the discussion I was making, I understand why it may seem like goalpost moving. But in reality, I was deliberately trying to tie the two related posts together because (IMO at least) they are related.
If your absurd claim that Waymo's trials provide just three relevant data points is not part of "the point", then why did you make it? It does not give us any confidence in the proposition that "the point" has been well thought-out.
Furthermore, "the point" keeps shifting: recently it shifted to raising doubts about the provenance of the data whipped up from a six-year-old article. At this point, I feel that a quote is appropriate: "the ability of people to consistently miss/twist the point to fit their own predetermined viewpoint is tiresome."
And in the vein of trying to steel-man your position, I gave the comment to ChatGPT to see if it, too, considered the central point a claim about AV having a "death wish." Here's what it said:
"The statement highlights that some AI tasks, while effective, may deviate from conventional practices. An example is given where the Department of Defense (DoD) employed a company to train a dogfighting simulator using reinforcement learning (RL). Pilots were surprised by the simulator breaking established best practices and behaving recklessly, akin to a pilot with a disregard for safety. The implication is that while such behavior might be advantageous in a military context, it may pose risks or be unsuitable in civilian settings, such as public roads. The statement underscores the need to carefully consider and tailor AI applications to specific contexts and objectives."
So it seemed to recognize that the central point is that "the behavior" in question is "breaking established best practices" and that the "implication is that while such behavior might be advantageous in a military context, it may pose risks or be unsuitable in civilian settings". There's probably some irony in the fact that AI did better at a reading task.
Really the only thing left is for you to take a flight to SF and watch the Waymo cars drive. Or ride in one if you dare.
This dispute is not over a difficult or subtle issue: all the people who have responded in this thread see clearly the obvious and unequivocal reading - and, in your own example, ChatGPT also does! It has identified the tacit subject of the sentence "Possibly good in war, maybe not so good on a public road" as the specific military system, with behavior explicitly described as like having a death wish, that is the only topic of the preceding sentence. For one thing, the phrase 'possibly good in war' makes no sense in the reading you are trying to pass off: why, out of nowhere, did the needs of the military appear? And unexpected behavior is not something desirable in general in military systems, any more than elsewhere - it would take very special circumstances and a specific sort of behavior for that proposition to even be entertained. We can see, therefore, that dcow was right to respond 'your comparison is RL dogfighting?...' and 'Modern AVs are not driving like they have a death wish by any stretch of the imagination...'
Oh, and next time you invoke ChatGPT's response, include the prompt, verbatim, like this for example:
Prompt: In the statement 'That’s true, but also one of the selling points of some AI tasks. As a non-hypothetical example, the DoD hired a company to train a software dogfighting simulator with RL. What surprised the pilots was how many "best practices" it broke and how it essentially behaved like a pilot with a death wish. Possibly good in war, maybe not so good on a public road.', what is being called ' not so good on a public road'?
Response: In the statement, the phrase "not so good on a public road" refers to the behavior of the software dogfighting simulator trained with reinforcement learning (RL). The implication is that the simulator, which exhibited behavior contrary to conventional "best practices" and behaved like a pilot with a "death wish," might not be suitable or safe for use in a public road scenario. This suggests a concern about the potentially risky or unpredictable behavior of the AI system in a real-world, civilian setting such as driving on public roads.
What we have in this discussion is a motte-and-bailey fallacy, as we can see in your response to my first post here, which was:
None of these three cases involved the Waymo car behaving in ways that are not that uncommon among human drivers, and our theory of mind does not make us nearly-infallible predictors of what another driver is going to do. Your objection becomes essentially hypothetical unless these cars are behaving in ways that are both outside of the norms established by the driving public, and dangerous." Your reply, in outline, goes like this:
> That's true...
Here we are in the motte, where you nominally accept that the relevance of your concern, which is not unreasonable in itself, is constrained by the extensive testing that has been performed by Waymo so far...
> ...but...
Here we enter The bailey, where we are supposed to turn our attention to an unrelated system, which was found, on testing, to have alarming unexpected behavior. The bailey has become a place where Waymo has been curating the data to the point where we simply don’t have the slightest idea whether there’s dangerous behavior outside of the human norms lurking in Waymo cars.
It has also become a place where all of Waymo’s extensive testing has produced just three data points against this view. I must say that it seems generous of you to concede even three, if Waymo is curating the data to the extent you imply.
This is where you are bypassing the point about how much faith we can put in the data, because we don't have the full results of their extensive testing. We only have the results they are willing to share.
As an analogy, I have a relative who loves to talk about all the times he's won money gambling on the craps table. I almost never hear any information about his losses, unless they are couched to say how much more money he's won later. So would you say I can conclude he's an expert gambler who should quit his job to play craps full time, or do you think there might be some human bias in reporting going on that I should be skeptical about?
At least the idea that Waymo is hiding data that would reveal the vehicles as being too dangerous to be on public roads is not quite that wrongheaded, but it implies that Waymo is doing this on a massive scale that cannot possibly succeed in the long run. It is not clear, for example, how it could hide information about similar or worse incidents from the insurance companies, and a story about your gambling relative is not changing my mind.
This is your dichotomous thinking again. I am not making a point about it being "safe" or "unsafe" as a dichotomous choice. I'm saying we don't have good data to make a claim one way or another. That's also different from saying, "we have no information at all." When you combine that with the fact that we have other evidence that RL models can result in unpredictable behavior, it should give us pause.
Insufficient data + priors about unpredictable behavior = uncertainty of performance
The through-line of my entire point is that uncertainty erodes trust, and trust is necessary for wide adoption in the public sphere. It isn't a hard concept if you can lay your bias and dichotomous thinking aside to consider it. Waymo seems to agree; although the PR headline is about 7MM miles traveled, the report says:
>"the required ADS VMT to establish statistical significance ranges from tens to hundreds of millions of miles, and the fatal outcome requires hundreds of millions to billions of miles of driving are needed."
So unless you have better data to show, you've demonstrated nothing to make me change my mind.
You don't get a thread of 173 posts (44 by you) over 6 days and counting (4 in the last day) over the anodyne position you are now professing. That time has been spent dealing with all the bogus stuff that you are now tacitly disavowing, such as the bizarre three data points claim that you brought up again as recently as your previous post.
> ...change my mind.
You have been tacitly changing your mind throughout, though, of course, you will not admit it.
It certainly comes across like either you're too hung up on a position to read critically or you got fooled by a PR piece without knowing how to properly interpret it. I guess that's a win for the SV hype machine.
On the one hand, I can understand that it is difficult to keep track of everything you have said throughout your twists and turns, but on the other, it seems particularly important one to keep this one in mind as, if verified, it would be far more damning of Waymo than everything else you have said, combined!
I've replaced your word "it" to make the point as clear as possible.
>"And you chose to quote it, and to make the point that" [unexpected behavior] "is not the sort of thing we want on public roads"
Or to put it different terms, the sometimes unpredictable behavior resulting from RL may be a feature on the battlefield but a bug on public roadways.
I still stand by that point. And, yes, that means dcow also missed that point. There's nothing hypothetical about bringing about real-world case studies of autonomous behavior based on RL models. We've been through this so many times now that I'm coming to the conclusion you may be arguing in bad faith or you get incredibly distracted by certain terms that it inhibits reading comprehension.
>"And you chose to quote it, and to make the point that" [unexpected behavior] "is not the sort of thing we want on public roads"
Ah, so we're playing the "I didn't say what I just said" game now. Here's what you actually wrote:
"That’s true, but also one of the selling points of some AI tasks. As a non hypothetical example, the DoD hired a company to train a software dogfighting simulator with RL. What surprised the pilots was how many “best practices” it broke and how it essentially behaved like a pilot with a death wish. Possibly good in war, maybe not so good on a public road."
Anyone with the slightest familiarity with language will find no credibility in your claim that in this comment, the thing that is being called "possibly good in war" but "not so good on a public road" is anything other than the one explicitly-mentioned aspect of the system that you have specifically chosen to present as an example. Furthermore it was completely reasonable for dcow to respond "modern AVs are not driving like they have a death wish by any stretch of the imagination" after you chose to make the above statement.
With your latest response, you continue to put more weight on an anecdote about this unrelated system than you do on the extensive empirical evidence from testing the actual system that is the subject of this article.
They're telling you what you wrote, not what you intended to write.
They're correct.
> The "explicitly mentioned aspect" is the unpredictable behavior.
No, it was breaking best practices and behaving like it had a death wish. That's something that has some overlap with being unpredictable, but is not at all the same thing. You can have a predictable death wish, even.
> maybe your biases made you infer more than what was actually said
I don't think I have much bias here and I agree with them. Also consider that the person that writes something is biased to think the communication was clearer than it actually was.
> And not to beat a dead horse, my secondary point has consistently been
It's neat to have a correct secondary point, but it won't make your primary point correct, and people don't need to add a disclaimer of "while your secondary point is fine" every time they criticize your primary point.
Now I'll concede it's possible that RL can lead to finding new best practices. But that's not the case in the story relayed. It is only a good practice for the scenario because it was an unpiloted aircraft because it otherwise puts the pilot at too much risk (which is why the pilot say it was acting "as if" it had a death wish. He was anthropomorphizing it.) That still means it's bad practice when the goal is to save human lives.
But that's not the same as being unpredictable. Those are orthogonal complaints, and nothing in your quote suggests that the "death wish" flying was more unpredictable.
And we already know that these cars act super cautious, the opposite of acting like they have a death with.
So that quote has no useful information that we could transfer to the car situation.
What I am saying here should be clear to someone such as yourself, who is invoking our theory of mind in his claims: I am making a distinction between what you are now saying you meant all along, and what other people will recognize as having been your intent when you first wrote the passage in question. We cannot present proof, but nevertheless we know, beyond reasonable doubt. Your explanation does not pass the sniff test.
Wouldn't the most important safety question be "whether Waymo’s technology makes fatal crashes MORE likely?" Why assume Autonomous Vehicles have to perform better?
Waymo less fatalities == Win
Waymo same fatalities == Draw
Waymo more fatalities == Bad Waymo
A significant proportion of the current fatal accident rate is based on incidents which involve driving so bad that humans implicated are severely punished for it, or at least banned from driving for a period.
Why would anyone set the benchmark for a commercial driving company lower?
Also, we've accepted and made legal frameworks around the concept that people sometimes kill each other with cars. Robots killing people with cars does not benefit from this carve out.
I am neither, hence I will not trust any system that does not perform at least as well as I do. The bar is much higher than the average.
...which is a pretty big fallacy and, if implemented as law (which I am working to make sure it is not), will get a lot of people killed.
Literal drug addicts, people driving under the influence of alcohol, and people driving distracted are driving right now. Even AV that performs at the level of the median driver will massively reduce the number of those people actually making driving decisions, and therefore directly reduce road deaths.
Why? Because most accidents and deaths are caused by drivers performing at the low end of the spectrum. In the US, a full third of accidents are caused by drunk driving - which would be virtually nonexistent with AVs ("virtually" because the AVs aren't going to be perfect and are going to need manual intervention).
The bar is literally the average, because if everyone drove average, deaths would plummet.
And, unlike the American public, it's massively easier to improve AV driving behavior than average human driver behavior. A system that starts out eliminating the lower end of driving performance while improving over time is a deal too good to turn down if you care about human life.
It remains to be seen if autonomous driving systems are actually safe.
Alternative example:
It remains to be seen if microplastics will cause higher rates of cancer.
It remains to be seen if organic peanut butter will cause lower rates of cancer.
ML having completely unpredictable failure modes is like the entire case against putting them anywhere. What would you call a vision system that mis-identifies a stop sigh because of a couple unrelated lines painted on it, other than "unpredictable"?
There's a good reason for this. It's because the human can be interrogated into what was going through their mind whereas many ML models cannot. That means we can't ascertain if the ML accident is part of a latent issue that may rear its ugly head again (or in a slightly different manner) or just a one-off. That is the original point: a theory-of-mind is important to risk management. That means we will struggle to mitigate the risk if we don't "hyperanalyze" it.
(Linking to the web.archive version because the graphics are better / more understandable when in the context of some of the text)
Chapter 6 is the most relevant here, but it's all a thought provoking story.
Cars can do unexpected things for good reasons, as the AI did in this case.
A good indicator of their next move is their previous move.
> "Acting unexpectedly" is one of the aspects that makes dealing with mentally ill people anxiety-producing.
Wat.
Depends on the size of the tree branch.
But also, being hit from behind at low speeds was a pretty common thing in early testing in Mountain View, when they were using the Google name on cars. That there are only a handful of incidents reported in this report means either the software has gotten better at communicating its intent to other drivers, or the driving public is aware that Waymo cars are way more cautious --- if they only make lane changes and unprotected turns by engraved invitation and everyone knows it, that's fine too.
In the early days, it seemed like it might be appropriate to install a 1979 regulation 5 mph rear bumper on these cars, because they'd likely get hit often enough.
There's at least one other alternative: selection bias. Since there is no standard industry definition, companies are allowed to not report many incidents.
>"Waymo, on the other hand, ran complex computer simulations after each disengagement, and only reported to the DMV those where it believed the driver was correct to take charge, rather than being overly cautious."[1]
So it may not be reported unless it meets Waymo's (non-independent) selection criteria. I think most people can at least recognize there is a potential conflict of interest when objective reporting isn't required.
[1] https://spectrum.ieee.org/have-selfdriving-cars-stopped-gett...
Our thinking fast reflexes are to pretty much avoid any obstacle whether living or dead. Hopefully our thinking slow brain has time to do a proper evaluation before doing anything drastic.
[1] https://www.kqed.org/news/11801138/apple-engineer-killed-in-...
[2] https://abc7news.com/tesla-autopilot-crash-driver-assist-cra...
Depends on the size of the tree branch.
As a thought experiment: Let's put you in a driving simulator, and ask you re-drive the same scenario 10,000 times. In each scenario, we randomly change the size of the branch -- both weight and volume. Repeat the same test with 100 other drivers. Repeat the same test with 100 different AV algorithms. I am sure each driver would have their own definition of "branch too large to safely drive over".
What concrete point are you trying to make?
being hit from behind at low speeds was a pretty common thing in early testing in Mountain View, when they were using the Google name on cars.
From driving near the things, they're very cautious and sometimes start to move and don't (I saw one try to make a lane change for about a mile on el camino before it was able to, turning the blinker on and off the whole time).
>> in early testing in Mountain View, when they were using the Google name on cars.
> Early testing? From what perspective.
You know, before they switched the name to Waymo.
On the road, sometimes I feel like I'm the only one following the rules. :/
>"In August, a Waymo at an intersection “began to proceed forward” but then “slowed to a stop” and was hit from behind by an SUV."
>"Wat."
Are you saying you don't understand why unexpected behavior causes anxiety? It's a pretty well documented effect, from rats to humans.
> > "Acting unexpectedly" is one of the aspects that makes dealing with mentally ill people anxiety-producing.
> Wat.
The "Wat" probably refers to the fact that this seems unrelated. Dealing with mentally ill people is anxiety-inducing because they act unexpectedly... so what? Lots of things are unexpected. They shouldn't all be drawn into the analogy that says "well, that thing, plus a load of other things, can induce anxiety, therefore that thing should be tarnished with the same brush as all of those things."
People drive in unexpected ways all the time. Of all the criticisms to level at them, "the thing you're doing plus a load of other stuff can induce anxiety" probably isn't top of the list.
Would also be nice to see all incidents, not just injury incidents to see what kind of unexpected "mistakes" these cars are making
I think you're understating it.
Mentally ill people act far off the cuff. If I'm walking outside, people can behave unexpectedly (but within the parameters of behavior that doesn't make me anxious).
Imagine: stopping instantly to bend over and tie one's shoes or to look at a storefront. Taking up multiple spaces on the sidewalk. Dropping an item that causes a loud noise. All of these are unexpected movements that require a reaction.
However, if somebody screams about CIA conspiracies or has very erratic mannerisms, that would create more anxiety.
So, apparently their point is that AI behavior on the roads might be more jarring than normal jackass human behavior on the roads.
So to put a finer point on it, people need to acknowledge that public trust is necessary to wide-scale adoption of AV tech. Plenty of psychological research shows how we aren't intuitively wired to understand statistics. So all the bleating about statistics may be necessary, but not sufficient, to get wide-scale adoption of AVs on public roadways.
It wasn't just the word anxiety; that's a mischaracterisation, and another psychology-adjacent misstep with "hung up on". Your point wasn't great, or at least was very poorly articulated, and that's what caused me to "miss the real point". You seem to have clarified it with this comment, so my challenge to your original point seems to have helped.
I'm saying that your leap from "the car in front is behaving unexpectedly because you don't have all the information it has" to "a bunch of mentally ill drivers" is such a complete non-sequitur that it's deserving of a Wat [1].
Why do mentally ill people cause some people anxiety? Because they may behave erratically.* In other words, we have a lot more difficulty ascertaining what they are thinking, and by extension, predicting their behavior. The same can be said for the general public sentiment towards AVs.
* I also understand "mental illness" is a blanket category and I'm not using it as a stigmatizing term. It may be stretching the analogy too much, but it's simply a proxy for "I have a hard time predicting what's going through this persons mind"
I imagine the collision speed was quite low since they just left the junction. There's a reason things like following distance and looking in front of you while driving™ exist.
If humans can't do that, the solution is probably more automation.
The Uber fatality from years back showed that the software used "action suppression" to mitigate nuisance braking. The idea that that would be acceptable on a safety-critical software application should give us pause to consider that more automation is the knee-jerk solution.
Now, couple that with: 1) a general approach of "when in doubt, brake" and the hacky workaround to suppress braking if it's occurring too much, and you've got a bad nuisance braking problem that's primed for a safety incident.
But again, this is disjointed from reality. I'd agree that if we all left the recommended 4.5 seconds of stopping distance, we'd all be safer. But that's not how our roads were built and that's not how humans drive. You're tilting at windmills here to make a point that doesn't need to be made because it doesn't apply to reality.
Oh, it emphatically is how our roads were built. Most miles in the states were still laid down in an era where cars didn't do more than 45. As for how humans drive... I'll direct the audience's attention to the 3,700 per day road fatality rate in the United States. Even if they brake check more than human drivers, an automated vehicle being forced to follow its programming to maintain a healthy follow distance may very well save lives, especially in an ecosystem where they are the dominant vehicle on the road as opposed to human operated vehicles.
I'm not scared of mentally ill drivers. I'm scared of rich 16 year olds. I'm scared of drunk drivers. I'm scared of drivers sitting so low they can't see most of what is happening around them. I'm scared of drivers having seizures while driving (my mom was hit a while ago by a man who lost his license due to seizures, and still refused to stop driving). I'm scared of drivers who drive without a license because "fuck them, I drive when I want to". I'm scared of people mixing up gas and break pedals (just got hit by one), in cars which can go 0-60 in 2.6 seconds weighing 6000lbs.
This specific problem is just as much a design flaw as it is a PEBCAC issue.
You have two very similar pedals that perform polar opposite functions right next to each other, and they are both operated by the same foot.
I'm surprised this isn't a bigger problem.
man who lost his license due to seizures, and still refused to stop driving
Another one, suspended license, got drunk, and decided to play pinball with the car my fiance was in, causing lifelong injuries.
You can't get away from those. America is just a fun place.
You can't get away from those.
If these tragedies you describe keep happening (as described), and the fault you identified points directly to the problems of American drivers, I find it shocking you suddenly describe it as "this one issue". Either it's as problematic as you described or it's not.
And if one believes that, e.g. self driving cars will do a lot to address it in the next decade, they especially might not want to incur the costs of the alternatives.
I've seen people reverse up a highway onramp.
How will AI drivers navigate these 'cultural' differences? Will they insist on following the letter of the law (presumably, for liability) or will they adapt to local practices and potentially lower their overall collision rate with humans driving poorly. Interesting near term question.
The unexpected is, by definition, what was not predicted.
"all the time" means it is not unexpected. Humans do mistakes and omissions that you've subconsciously learned that humans do, so you can adjust to deal with that.
The difficulty with AI is that it behaves like a completely alien being, making errors that are alien and unpredictable to a human driver.
For example, just because someone has their turn signal on and is slowing down doesn't mean they're going to turn.
From my experience with LLMs, AIs even when called to be creative are fairly consistent, even if at first they seem to be producing output that’s more creative than human norms.
Perhaps a self driving car's unexpected behavior is on the "too defensive" side while human drivers are on the "suddenly do something wildly dangerous" but this is easier to account for as a driver rather than more difficult to account for.
You wouldn’t expect someone to suddenly accelerate from a stop while they have a red light, so you don’t typically look both ways when driving through a light that has already been green for a while when there are cars on either side waiting.
I’m not saying Waymo cars do that; just an example of expected unexpected vs unexpected unexpected behavior.
Also sometimes I can intuit that the driver in front of me is probably going to do something dumb because of other minor things I've observed about their driving style.
are we going to jail some programmer because the missed a corner case and created a fatality.
ultimately what s problematic is what capitalism usually does which is externalize the costs via techno washing.
[1] https://www.motortrend.com/news/volvo-will-accept-full-liabi...
"Acting Unexpectedly" can often mean following the actual laws and general guidelines for safe and/or defensive driving. I would hazard a guess that sometimes doing the intuitive thing is, in reality, unsafe and/or against the law. If the car does this in 99% of circumstances, and still gets rear-ended, who is really the problem here?
2. Most of the time it is a false perception. You only notice the people whizzing past you, not the people driving the speed limit along with you because they never pass you (a variant of survivorship bias). This of course depends on where you are, but most people in fact do drive around the speed limit.
Counterpoint: Google Maps thinks you can drive an 86-mile trip from from Springfield, MA to Albany, NY in 80 minutes, a route which is patently impossible if you're driving the speed limit; you cannot drive 84 miles on I-91 in 76 minutes, an average just over 66 mph, on a road with no segments 65 mph segments, without exceeding the speed limit.
https://www.google.com/maps/dir/Springfield,+Massachusetts/A...
But also, this claim just doesn't stand up to scrutiny. Cars take up a lot of space. You can't just drive through them, you have to take affirmative action to change lanes and pass them. You'd find yourself behind people driving the speed limit pretty often, which you will notice.
That is no counterpoint. I am talking about people, not Google Maps. Also, the standard freeway speed limit is 65 MPH AFAIK, so I-91 seems like an exception that Google Maps is not accounting for (and perhaps human drivers as well).
> You'd find yourself behind people driving the speed limit pretty often, which you will notice.
I assume you're implying that you're always driving above the speed limit and you and you're saying you don't find yourself passing people. There could be many reasons for this. You could be in one of the exceptions that I mention, or you could be driving at times and/or along roads without many people. You could also be selectively remembering things; you are much more likely to remember passing people if the incident is frustrating and less likely if it is not frustrating.
You cannot average 66 mph without going over 65 mph, and the states involved don't post speed limits over 65 mph. You cannot get this result with by a simple oversight under the assumption that speed limits represent the speed people actually drive.
Consider what inputs might lead Google Maps to such a conclusion.
Do you care about justification, or do you care about being safe?
I go by feel. If it feels much safer to speed, I do. Otherwise I'm in the right hand lane, usually a safe stopping distance behind a big rig for aero gains.
The unfortunate thing is people take your propensity to maintain a safe follow distance as an invitation to cut aggressively in front of you, potentially across multiple lanes.
It sucks all around, but I don’t think AVs are the solution and I definitely don’t trust any company producing AVs is delivering a product that does what it claims. Waymo doesn’t acknowledge in this about the part where their vehicles have human operators take control when the car doesn’t know what to do. I assume they’re not including that data. That’s going to skew the result. Even if the car is super safe in most driving conditions, ignoring what is arguably the least safe conditions the car can be in in your data analysis is fucked and intentionally dishonest.
intuit the behavior of other humans through the theory-of-mind
If AVs consistently act "unexpectedly"
And I've been to MANY places in the US where people drive terribly and completely unexpectedly. I've seen people put on a left turn signal then go into the RIGHT lane. I've seen someone pass on a one lane overpass, where the car in front was towing a heavy load
What's good about these cars is they will ALL drive consistently, and the more people get used to them the easier it well be to drive with them because you will know how waymo cars react.
One of the first thing my driving instructor told me was: never trust anyone on the road except yourself. That’s why you see people waiting for a car with the indicator on to visually hint they really intended to turn.
Yes, but...there is something else to be said here. One of the things we have evolved to do, without necessarily appreciating it, is to intuit the behavior of other humans through the theory-of-mind. If AVs consistent act "unexpectedly", this injects a lot more uncertainty into the system, especially when interacting with other humans.
"Acting unexpectedly" is one of the aspects that makes dealing with mentally ill people anxiety-producing. I don't think most of us would automatically want to share the roads with a bunch of mentally ill drivers, even if, statistically, they were better than neurotypical drivers. There's something to be said about these scenarios regarding trust being derived from understanding what someone else is likely thinking.
Edit: the other aspect that needs to be said is that tech in society is governed by policy. People don't generally just accept policy based on statistical arguments. If you think that you can expect people to accept policies that allow AVs without addressing the trust issue, it might be a painful ride.