No, that isn't a problem with numbers. That is a problem with letting your imagination run things.

The scenarios and questions you ask are easy to explore in the data. If the number of crashes goes down, contrived scenarios don't really impact things. If they stay flat or go up, then you have to answer why.

To emphasize, for your imagination here to be relevant, the number has to do one of those. If the number goes down, even if it shifted some situations into a wreck that would not have been before, it must have done fewer of those than it moved wrecks to non-wrecks. If it somehow causes more situations to be wrecks, the number, by definition, goes up.

The question is whether anyone is actually interrogating the data in that way, not whether it can be interrogated. Large data-driven institutions are notorious for taking a narrow view of things.

The problem is precisely a lack of imagination, or perhaps more accurately, critical thought.

The parent is precisely engaged in the kind of interrogation that is necessary. If you are aware or an analysis that bears on his questions, now would be the time to share it.

This feels like sophistry to hinder progress. The OP is literally about studies done. Did you find problems with them? Or are you only imagining problems there could be?

The latter, which is why the actual studies would be most welcome.

But more to the point, there is a track record of narrow interpretation of studies and missing of the big picture. It’s hardly sophistry, it’s critical thought.

Is there a track record for this specific organization? Or there are a lot of publicly known tech companies that have missed the big picture?

I disagree. I've been a licensed driver for over 40 years. I had one minor fender-bender at 16 years old, learned my lesson, and haven't had another accident since. I'm an exceptionally good driver as evidenced by this, plus the fact that I've taken performance driving classes, simulators, and so forth.

I, personally, am a much lower risk than the typical driver. Putting a drivers' aid device in my car may well increase the danger to me and those around me, even while having it in other cars decreases danger.

BTW - my car's traction control is evil. It's a rear wheel drive car, without limited slip. So when I pull out onto the road from a driveway, if there's the slightest bit of sand on the road, I get enough spin to engage the traction control. The car deals with this by vastly cutting engine power. In Texas, where the regular surface road I'm pulling out to is commonly 60mph, and often faster, I need to accelerate quickly to avoid the guy approaching - yet the traction control has just left me wallowing, dead in the water, unable to accelerate strongly. Except in wet conditions, disabling the traction control is always the first thing I do when I sit in the car.

I thought I was a low risk driver, then when doing a lot of hours riding a motorcycle it became very apparent how incredibly terrible of a car driver I was. So much general threat and risk was invisible as a car driver contained in a big metal box.

I do a lot of cycling and it's amazing how much more aware you are when any minor bump is going to cause you injury. I've avoided accidents multiple seconds in advance because something about the way someone was driving made me nervous, it's like you get a sixth sense.

visibility is also dramatically better on a bike

Riding a bike (motorcycle or bicycle) should be mandatory for learning to drive a car.

Every person on the planet believes they’re an above average driver. No one thinks they need these types of safety features in their car because they’re a safe driver and the problem is every other driver.

It terrifies me that people are so oblivious to their own flaws that they would willingly disable features such as AEB if they could. I’m extremely thankful for regulations that prevent such life-saving safety features from being disabled.

An average person has 3-4 car crashes in a lifetime. Having 1 minor accident in 40 years of driving is above average.

The only time I ever came close to having a crash is when lane assist tried to drive in the dividers on a highway with construction going on, so lane assist is the first to go in a new car.

The average person also has their wrecks at the beginning and end of their driving. Having 1 while in late 50s to 60s isn't that unlikely, either. And sets you on the average track.

> Every person on the planet believes they’re an above average driver

Not everyone. I don't think I'm an above average driver at all.

Even if you would rather not have the feature in your car, wouldn't you rather the idiot about to hit you had it in his?

Unless you are proposing that they should require it in everyone's cars except yorus.

> If the number of crashes goes down, contrived scenarios don't really impact things.

Yes they do impact things.

Consider a pill that, given to a wide population of test subjects, reduces heart attacks by 20%. That's great! But when you look at the data more closely, it strongly reduces the risk for people older than 40, and it increases the risk for people under 40. Should you prescribe that pill to everyone at risk of heart attack? Of course not.

Does AEB have a pattern like this? Who knows! Someone should test or cite a good test!

TFA links a meta analysis. With a 38% impact, that is going to be tough to argue against. So... Consider it cited, but you aren't looking?

You're not listening. The 38% number can be true and there can be particular groups that are hurt more than helped. Nobody is trying to "argue against" the 38% number.

In a cost benefit, if the number drops 38%, you are going to have to work to show it isn't worth it. What would the number have to be?

Can they make the system even better by finding cases it didn't help? Almost certainly. Progress isn't a straight line, but increases are increases.

> In a cost benefit, if the number drops 38%, you are going to have to work to show it isn't worth it. What would the number have to be?

Where did anyone say "isn't worth it"? The sentence I see is "The problem with numbers is that they don't tell the whole story."

Though in a more general sense, if someone does invent a system that has strong average safety gains, but makes the majority of people less safe... that's a hard sell.

Fair, if you are just pushing that we need to keep looking at the numbers and to be aware of some ways it can go bad, fine. I took the topic here to be that these studies were bad. In a known and showable way.

It isn't a binary choice, you can disable the function for situations it's not suited for. Nothing should be forcing us to enable harmful scenarios just because.

well, the law explicitly says mandatory. Defaults tend to stick.

Respectfully, you're taking an extremely naive stance to a topic where others are trying to bring nuance. I think if you listen carefully, you'd learn something that will help you not just on this topic but in your life.

Logical fallacies like the base rate fallacy are real, and they fool all of us frequently, myself included.

https://en.m.wikipedia.org/wiki/Base_rate_fallacy

Bringing nuance is fine, but it is also an easy stalling tactic in progress. Clear improvements in safety are shown in the studies linked here. From an organization that has shown remarkable progress in safety over the years. And has been beset with such "nuance" since the beginning.

Literally the same arguments were and are made against seat belts. Often. And is why I am not too willing to give way here.

Now, if you are just saying, "I hope they keep their eyes on the data and are willing to keep improving," great. I agree. If you are trying to angle it that this is not a safety improvement, I will loudly disagree.

I think it is probably overall a safety improvement, and I think many of the arguments about losing control of your car are a bit overblown. But, there's something to them, and it rubs me the wrong way to ignore and dismiss those you disagree with. It's too easy to characterize a position you don't agree with as one that is operating in bad faith or ignorance.

> The scenarios and questions you ask are easy to explore in the data.

This sounds like the beginnings of the McNamara fallacy: that there is some abstract "data" where all the relevant variables can be explored to decide the best course of action. The reason it's a fallacy is that data is itself only a reflection of conscientious decisions about what to measure and how to measure, and these are invariably limited to what we already appreciate and understand, not accounting for the things we don't know how to measure and don't know we should be measuring.

To be more concrete, let's say we follow your suggestion and focus on crashes. Maybe it's found that automatic breaking reduces the number of crashes so it's mandated. But maybe automatic breaking also increases the likelihood that a crash is deadly in some sub-population like pedestrians, construction workers, and motorcyclists. So overall we see crashes decline because a large number of minor crashes are averted but we also see deadly crashes increase because pedestrians, construction workers, and motorcyclists are killed more frequently.

So data isn't a free lunch. Every decision about what to measure and what to optimize involves some prior contextual understanding. A misunderstanding of the context means the data will misrepresent some aspect of the problem and you'll overoptimize yourself into a local maximum.

Again, you are contriving bullshit to scare folks. And you didn't look at the studies. 38% decrease in accidents along with decrease in fatalities. In the data. For the worries you are sprouting, the data would have to look very different.

It's not "contriving bullshit" to give some examples of how a single average number could be misleading, clearly labeled as hypotheticals.

It is literally the definition of FUD here, though. These aren't things that have happened. And the data is ridiculously clear on progress. To paint these doubts is pretty much only to spread fear and uncertainty.

It's not FUD, they're not stating it as a fact, they're asking for an analysis of AEB in lots of different situations and not just overall.

That is how many FUD tactics are done. It is a rhetorical trick to try and maintain an exit...

Yes, but sharing a characteristic with FUD is not enough to make something FUD. It's a realistic concern.

Someone else mentioned airbags. Airbags are great overall but increase danger for some people, and that's without obvious signs of anything being wrong.

With AEB we know it makes dangerous mistakes in significant amounts. So we very much should analyze different implementations under different kinds of weather, road, and driving style.

We don't know it makes dangerous mistakes. Randos on the internet assert that. The data shows fairly convincing evidence is reduces wrecks.

Is it a new behavior and jarring when engaged? Certainly. Should we keep watching the data for problems? Certainly. Should folks report all problems so that they are tracked? Certainly.

Do most people actually have trouble driving safely? Despite their insistent view of how great they are at driving? ...

Do I need to go find dashcams? We know it makes dangerous mistakes. Come on.

This is not one I will cede without evidence, so yeah. With the amusing extra burden that I have yet to know a safe driver that used a dash camera.

I have been in a car where the alert went off that it was engaging aeb. The driver was far too close to the car in front of us...

Let's see, what can I find on youtube in a couple minutes of searching. In increasing order of wrongness:

https://www.youtube.com/watch?v=BLRRBHNjCn4 There's a car that it's reasonable to worry about, but it was clearly braking and almost stopped by the time AEB kicked in, and AEB kept going as that car went to zero.

https://www.youtube.com/watch?v=TO6tJnM_HSs Adaptive cruise was already in control and working fine.

https://www.youtube.com/watch?v=hCk6yrN453c Absolutely nothing in front.

And whatever happened here probably counts: https://theintercept.com/2023/01/10/tesla-crash-footage-auto...

I think they have a great point though, accurate measurement of the types of incidents is just as important if not more than the total.

This conjures the fear that the agency in question is bad at this. Do you have evidence that they are? Can it happen? Yes. We could also have it so that no wrecks from minorities are counted, such that we are blind to how they are impacted. Would be terrible. No evidence that that has happened.

> This conjures the fear that the agency in question is bad at this. Do you have evidence that they are?

Yes. This was my movivation for questioning the "data says" logic. The NHTSA has made decisions based on data that were poor decisions because they misunderstood the problem domain and became overly reliant on specific metrics. Here is a copy-paste of an earlier comment in this thread:

--

> the NHTSA is a pretty cautious organization, and they like numbers.

They may be cautious but they're not necessarily smart. Their vehicle safety ratings only measure the safety of occupants in that vehicle. Result? An arms race where cars keep getting bigger, heavier, and taller to do better in crash tests against bigger, heavier, and taller cars. These "safer" vehicles are now more deadly to everyone else, especially pedestrians and bicyclists.

McNamara fallacy in action.

I'm at against bigger cars as you can be. Literally biking to places when I can. I am not clear that this is the nhtsa's fault.

That said, I could cede this point if you can show evidence of it in these studies. The org is a bunch of people, so I grant they could make mistakes. Without specific concerns, this is the definition of fud. You aren't giving criticism of these studies and actions, but seeding doubt on their data.

The confusion here is that some people realize they aren't "average". That is, while some policy or another is good on the whole, it can also be bad for them personally. For them, the chance of a wreck can actually increase. You're demanding that they sacrifice themselves to save imbeciles who probably shouldn't be allowed to drive anyway.

For such people, these policies suck. And it's strange that there are those that demand they either be happy about it, or at least be quiet about it.

Anyone that frames this as a sacrifice is an entitled person. Same as folks that argued against seat belts.

The number of crashes will go up because of this "safety" feature. And then we'll be stuck with this "safety" feature for decades till the govt gets their act together to undo what they did.

I'll keep my 30yo car with no computers as long as I can thank you very much.

That's true of basically any change isn't it? Even seat belts conceivably could result in some very small increase in the chance of death as a result of it being harder to escape a vehicle. Obviously the benefits far outweigh the risk, but then we are just arguing over where to draw the line.

They way you crash in terms of whether your seatbelt helps or harms doesn't depend on your life choices. Features likely benefiting drunk and distracted drivers and likely hurting aware ones are different.

I think you missed their point. It's not that most situations are improved and some get worse, it's that those situations might be unevenly distributed among drivers.

A seat belt is basically the same for everyone. It makes things safer when you look at all drivers, and it makes things safer for just about any subgroup of drivers.

But a buggy AEB might not fit that pattern. Imagine a system that reduces crashes by half, but then adds an extra X crashes per million miles. If someone's unassisted crash rate is 5X, then this system reduces it to 3.5X and is a great help. If someone's unassisted crash rate is .5X, then this system increases their crash rate to 1.25X and is a terrible trade.

> it's that those situations might be unevenly distributed among drivers.

I think that may be true in all cases - for example seat belts and airbags requirements may affect people differently depending on height and weight?

Well specifically the idea is that it's so uneven it might increase the danger for certain groups, and seatbelts are pretty good overall. But bringing up airbags is a great example. If you're short enough (or possibly if you're tall enough), a standard airbag is more dangerous than nothing at all.

Your claim seems to be the group who are not helped are "good drivers"? ... except that doesn't feel like a well-defined group to me, and also is a group that almost everyone would say that they are in.

Insurance companies have a pretty good idea of who the "good" and "bad" drivers are. It's a question of statistics.

I’d imagine this is exactly the sort of analysis the powers NHTSA’s recommendations. They collect huge amounts of data on this stuff.

As someone who does ML for a living I can definitely say that false positive data from the field can significantly improve the experience for those 1000. That is what should be standardized if we wanted to see systems that work for everyone.