Agreed - faking an emergency is highly frowned upon and I'm betting it's a crime in most places. There's even a similarity with the following aspect of the Geneva Convention, which is that pretending to surrender is so harmful to norms that it is literally a war crime. https://en.wikipedia.org/wiki/Perfidy

"In the context of war, perfidy is a form of deception in which one side promises to act in good faith (such as by raising a flag of truce) with the intention of breaking that promise ... Perfidy constitutes a breach of the laws of war and so is a war crime, as it degrades the protections and mutual restraints developed in the interest of all parties, combatants and civilians. "

Pretending to surrender is something that's shown a lot in Star Trek and similar shows, and Pirates of the Carribean, but in our real-life world of 2023 it's a UN war crime to pretend to surrender, and it's really not a nice thing to do because if some people sometimes pretend to surrender, then pretty soon no one will trust surrender attempts and people who wanted to stop fighting will needlessly be lost.

The idea of criminals faking a medical emergency is similar imho. If I were in a position of determining how much time robbers should serve in prison, I would personally recommend a much much longer sentence if they faked a medical emergency, for exactly this reason.

I saw a video showing perfidy committed by a Russian soldier in Ukraine. Several Russians were coming out of a building surrendering to some Ukrainian soldiers, when one of them came out with a gun and started shooting. All the Russians were killed in the subsequent firefight, because one of them ruined it for the rest. It may seem like a clever ruse in movies, but in real life it gets people killed.

It'd be funny if it wasn't so terrible. At this point it would be easier to list the war crimes not committed by the Russian army, they seem to be well intent on speed running the Geneva convention. One can only hope one day there's trials in e.g. The Hague to bring all war criminals to justice.

Russians just have to start working on advanced weapons and then they'll be excused from the tribunals like Unit 731.

> One can only hope one day there's trials in e.g. The Hague to bring all war criminals to justice.

I hope that as well, but I think neither the US, Israel nor China will stand idle next to a nuclear power getting dragged in front of the ICC - not with the numerous counts of crimes each of these countries could reasonably be charged with. The Israelis may be the only ones being able to get away with a lot of their stuff given that their opponents continuously enact crimes as well (e.g. hiding military command structures under hospitals and schools), but the US and China? They can't pull that card.

> It may seem like a clever ruse in movies, but in real life it gets people killed.

To be fair, the point of the ruse (in real wars; in heroic fiction this sometimes varies) is usually to get people killed, too; the question is just which people get killed.

It gets more people to die on ALL sides which is why it's considered a shitty thing to do, even if it's technically a "valid" tactic in some anarchic philosophies. The use of this ruse will cause people to stop accepting surrender which is BAD because it means that the people who die will be the average Joe and average Ivan in different countries who really don't want to kill anyone and just want to go back to their family and their life. These goodhearted people just want to go home and would gladly surrender. Imagine this thought experiment. I'll use your pen name dragonwriter as the name of a hypothetical super-human bandit / assassin / mercenary / gun for hire who sometimes does jobs like assassinations and Mission Impossible style burglaries of royal treasure. Imagine the DragonWriter bandit has no qualms about killing and so anyone who stands in his way gets shot with the intent to kill, AND now there are some rumors that a group of cops had DragonWriter surrounded and so he surrendered to them but then he pulled out a hidden sword and spun in a circle chopping them all in half. Now, any time DragonWriter tries to surrender in the future, his surrender won't be accepted, but what's even worse is: his action that bought him maybe a 5 second advantage (vs a million different other ways he could have won or fled the fight), and this tiny temporary advantage by the ruse has imposed a cost in exchange:

This one infamous sword fight has now poisoned the well with respect to the global norm. Next time there's some other totally unaffiliated hat-wearing super-powered bandit who's stealing the Ark of the Covenant or the royal diamond or something, if THAT bandit tries to surrender, there's a high chance that person might be killed, because their surrender is distrusted since DragonWriter changed the norm and now every fight between supers and cops is now on "shoot to kill, no prisoners, take no quarter" stakes. Imagine how many times a famous super like Batman would have been killed if the authorities never accept a surrender because the surrender norm is ruined. It can be repaired but that might take a decade or more to repair the trust that was broken.

With strong norms about surrender, red cross medics being protected, etc., we can have things like the miracle of the WW1 Christmas Truce which help the war to be a little bit little less of pure hell, for all the pure-hearted or indifferent people dragged into being a soldier, perhaps even drafted or press-ganged, to be the disposable cannon fodder for a bloody and disease and gas drenched war for no good reason more than rich politicians' egos.

https://en.wikipedia.org/wiki/Christmas_truce

These people just want to go back home to their wife or their mum and would gladly gladly surrender if given the chance, but that's not possible if someone ruins the norm! It's like, for example even if someone is a super-villain for example, they should really not take human lives of innocent people, or destroy irreplaceable cultural heritage sites, or damage/burn an ecological area like a rain forest, because those, once lost, are forever gone, and leave our world impoverished.

The number of criminals faking medical emergencies to avoid having to appear in court is pretty high. And of course three days after they're right as rain. It's hard to act against because of the risk of punishing someone that really needs it.

Faking your own illness is very different to faking an ambulance with lights and sirens racing through town.

I'm pretty sure they meant faking being a ambulance or a firefighter or a cop to rob someone.

Ah sorry, misinterpretation on my end. You are probably right.

But: fake cops robbing people is pretty common too. Unfortunately.

Even more unfortunate, real cops doing it also happens.

Yeah and I bet that it would become even more common if cars fell for it every time. A predictable victim that always for for the same trick in the same way is much more attractive than a human that may respond in many different ways, that reads the news and may become more cautious if there is a crime wave.

I think the point is that humans do make this judgement call. We know what a real emergency vehicle and what real emergency personnel looks like, and we know what a crazy person waving their arms looks like.

When some idiot in the opposing lanes is waving for me to take a left across multiple lanes of moving traffic, I know that I should ignore their idiotic traffic direction. I know that when there's an emergency situation with a fire fighter telling me to continue through a red light, I know that I should follow the directions.

I hope you're not suggesting like a hater that a Tesla with FSD (available next week, I promise) isn't capable of performing a snap psych eval of any random person standing at the side of the road.

I am suggesting that a car which follows all human gestures is not better than a car that follows no human gestures.

But then you have essentially enabled everyone to wave your car to a stop, which could be very convenient for certain types of people.

I think the simple truth is that we are still a long long long time away from fully autonomous cars to be safely deployed anywhere except on very specific paths that are mostly closed off to everyone else.

But they could be tremendously useful there, for instance driving buses or airport shuttles and that kind of thing.

So this is currently possible today: I can walk out on any street today and redirect traffic. With or without my reflective vest, drivers will generally listen to me. And self-driving vehicles must as well.

Note that when I am doing traffic direction with appropriate authority, most drivers will never have heard of the organization I'm a part of (a local CERT team), nor have any real way to vet the authority I have. The authority they can determine I have is mostly that I'm doing it and no police officer has come by to tell me to stop doing it.

(Note that CERT members do not and should not self-deploy, mind you, and if you choose not to heed our directions, we almost certainly can find a cop car to come let you know you screwed up.)

Also bear in mind, many people directing traffic aren't public safety personnel at all. Construction workers are private employees, events have their own staff which direct traffic in various contexts, including public roads.

There's no definitive way to determine that someone is authorized to direct traffic, cars just need to obey them.

Human drivers are smart, and will generally judge your appearance, confidence, and context to determine whether or not your actions are legitimate. And you likely always pass these tests.

Imagine a teenager, dressed in street wear, who jumps into the middle of a street to stop traffic in a way that has no explanation given the scene, while their friends laugh on the sidewalk as they observe.

Or a group of men in masks stopping a car in an alley as their accomplices surround the vehicle.

Any human driver will recognize these scenarios as illegitimate. It isn't just the act of directing traffic that is judged in isolation.

In basically every example you gave, you still have to stop until people move though. The result is the same, despite everything else. The only one that's a bit iffy is if you're being carjacked, is a bit of an exception.

Legitimacy isn't really the problem, it's the circumstance of a human being in front of your car that prevents you from moving forward, and beyond getting out of your car and talking to them, or just waiting, you don't have much of a choice in what options you have.

I agree with the intent of your point, but what's the alternative? You can't just keep driving if someone is standing in the middle of the road.

Those were two examples, but they're not exhaustive. Traffic direction can (and often does) happen without someone blocking the vehicle.

In the case of an illegitimate request, a human driver might question the request and take a different action instead.

Are you being directed to run a red light? If you judge that someone is maliciously asking you to do so, you probably will choose to remain still instead.

Are you being directed to stop? If you judge that someone is maliciously asking you to do so, you may choose to turn around, reverse, turn into a parking lot or otherwise remove your vehicle from the situation. You don't have to run people over to refuse a command to stop.

And if you were completely blocked from moving, you'd probably call the police.

My point is that "always follow hand signals" is trivially problematic. I would expect that if self-driving cars followed any hand signal they saw, you'd see kids doing it for the lulz on the sidewalk in front of their school. ..."traffic jam challenge". After all, someone is less likely to act maliciously towards a human than they are a machine.

Okay, so your point was more about the specific directions being given, than the presence of a person in the road. Overall I agree.

One reason carjacking is rare is because people don't foolishly stop for obvious carjackings. If an autonomous car has trouble telling the difference between a carjacker and a construction worker, then it will be much more likely to get carjacked.

My brother and his teenage friends redirected traffic into their school grounds to prove that they could do it. They also passed all 'sure, looks legit' tests.

Humans aren’t perfect, but their BS detectors are better than nothing.

The more important test scenarios are life threatening or unreasonably disruptive situations rather than harmless pranks.

PKE transmitters given to firefighters with some mechanism to cycle private keys on autodriving taxis weekly.

Nowhere near perfect, but that makes it a lot harder than just getting a spike strip or a hand gun or a demolition derby car. We’re talking about people willing to risk a decade in prison to “do criminal things to a car and/or passengers”. How much work do you think they want to put into disabling a car with a remote? Why not just wave a gun in the driver’s face at a stop light?

>There's no definitive way to determine that someone is authorized to direct traffic, cars just need to obey them.

Exactly! But there still is a way. You don't obey anyone in any circumstances because that would be dangerous. People have a feeling for it, computers don't yet.

I'd obey such traffic commands from just about anybody in plain street clothes, unless they looked obviously crazy or if I was in part of the world where car jacking is common. At the very least I'd stop and ask them what was up. Maybe there is some dangerous condition ahead and they're trying to warn me.

Does the computerized car understand the meaning of the words "Turn back because the bridge is out"?

I miswrote, my meaning wasn’t that I would stop for nobody, but that I wouldn’t stop for just anybody in just any situation.

Essentially the same that you said.

This is something that people are very good at judging but software isn’t.

I’m trying to say that until we have figured out AGI, autonomous cars driving among people are very dangerous.

I bet you’d eyeball it though, right? Like if someone was telling cars to stop, looked nervous, and had a hand in a large enough pocket to hold a gun, I might head in the direction they were sending people (depending on visibility I guess) but I’d probably not slow down.

> Note that when I am doing traffic direction with appropriate authority, most drivers will never have heard of the organization I'm a part of

Reading between the lines it doesn’t seem like there’s a lot of communication to emergency services etc on how to deal with autonomous vehicles.

Even the police seem to be operating under misconceptions about what the vehicles can and can’t do.

Isn't it the vehicle operators operating the vehicles while ignoring that there are things that the vehicles must do in certain scenarios to be suitable for operation?

So there's no monolithic entity that handles public safety in any given area. Overlapping jurisdictions of police, fire, public works, utility companies, construction companies, volunteer orgs, etc. all will end up on the streets for different reasons.

If self-driving cars cannot cope with this, they should not be allowed on public roads. Very little has convinced me they should currently be street legal.

>most drivers will never have heard of the organization I'm a part of (a local CERT team)

Will you have to file a CVE after the incident?

As a sysadmin and a volunteer responder, I do find the fact CISA and FEMA each have a CERT program that are completely different things pretty funny.

Community Emergency Response Team: https://community.fema.gov/PreparednessCommunity/s/welcome-t...

I think the potential issue is that the driver of a non-emergency vehicle could start gesturing in particular ways that could trigger the autonomous car to think it was an emergency vehicle.

So then you have to teach the car what all the various kinds of emergency vehicles look like, in every jurisdiction. That's hard enough, but then what happens if it's an unmarked police car, for example?

To spell it out, if an SDC Taxi stops for anyone waving frantically, how do you prevent robbers and rapists from routinely harvesting their passengers?

Exactly how you stop people from flagging a human driven taxi with plights for help and then robbing or raping the passenger. This doesn't seem like a problem specific to self driving.

Relevant xkcd: https://xkcd.com/1958/

A Taxi has a sober human driver with professional experience to handle such situations.

A piece of software with some cameras and actuators is really not the same.

self driving car AI challenge, run into the cross walk and start waving every car through

I agree with treat it as real. Then, if the car has an adult passenger you can ask them for their judgement (and responsibility for the outcome!)

If the car has no passenger. Then stop and move slowly as required/indicated. Sure, it may be a car thief, but without a passenger, it is simply another car theft.

> When you see flashing lights on the highway coming up behind you, do you think “I bet that ambulance is faking it so they can get home faster”, or do you just move over?

“You” in the specific or general sense? Because, my observations of highway behavior suggest that actual humans are…far from ideal at this.