What's the meaningful distinction between the robot ignoring or missing relevant information? We're not in a court of law trying to suss out intent or lay blame on a specific entity. The essay is about robots killing people, and supposing this will continue without government regulation. The people are killed either way...

We can't address and fix a problem if we fundamentally misunderstand it.

The irony is that we're in this problem in the first place because people are releasing problematic tech before it is understood in the first place. Humanity at large shouldn't be part of the experimentation process, and the essay is arguing that we need tighter regulations around that.

The essay suggests "Any regulation of industrial robots stems from existing industrial regulation, which has been evolving for many decades" & that regulations need to be clear about other robots as well. Interestingly enough, if we applied existing regulations to Tesla's Autopilot as we would to a driver who racked up as many fatalities and crashes (736 crashes and 17 deaths in the U.S. since 2019 [0]), they'd be behind bars long ago, let alone never able to drive again.

0 - https://www.caranddriver.com/news/a44185487/report-tesla-aut...

Intentionality is a central concept in AI philosophy - from debates between Dennet, Churchland and Searle going back to the 1980s.

Aat this stage I agree that we are dealing with "blind machines", with no more "intention" than a steam engine.

However, the central argument in Schneier's piece seems to be that robots inherit the intentionality of their creators.

If I create a machine that kills someone, my "intentionality" is fair game to be dissected in court, and ultimately I may be found guilty as if I had killed with my own hands.

That is the kind of high bar I think society expects, and is certainly one that should scare the crap out of robotics developers (aside: I have been one) to ensure the highest standards of engineering discipline and industrial ethics.

Nah. Idiots walk into machinery all the time. Put up signs, do safety training, yell "Stop!" and still Darwin takes his share.

It's definitely fair to consider the 'intentionality' of the idiot too.

> Idiots walk into machinery all the time. Put up signs, do safety training, yell "Stop!" and still Darwin takes his share.

It's still on leadership if they choose to hire idiots who get hurt. The solution is: don't hire idiots.

If you look historically at manufacturing and the origins of signage and safety training, you'll find them mandated by the government (i.e. OSHA, etc.) because those things weren't always voluntarily provided by the employer. Upton Sinclair opened our eyes to this 100 years ago.

What you're missing here is the numbers game. The size of the operation vs. the tolerance for worker injuries has a massive impact on this stuff.

When you run a shop with five experienced people, if each of them has a 1-in-a-million chance of getting seriously injured on any given day, each year you have a 1-in-548 chance of a serious injury. Any sane person would call this risk acceptable.

If you run a shop with 5000 people of equal skill level to those five above, you're talking a couple of deaths or permanent disability injuries per year, just from the numbers. Any rational person would say hey, that's the same risk, why are you mad? But society isn't rational and if any legal entity no matter how large kills or maims a couple of workers a day, there's an outcry and executive heads will roll. So yeah, they try to do better. At this point maybe 'hire smarter people' works?

Now imagine you're running a multinational company with 500,000 people. If you operate like that original shop you'll see a couple of hundred deaths in your first year and then get sued into oblivion or lynched, and remember those were skilled experts. There aren't 500,000 skilled experts available for hire, and most of these jobs are ahem less skilled anyway, so you're hiring people with much higher rates of personal injury. To operate at this level you need to do stuff that seems dumb to the uninitiated, like relying on layers of subcontracting companies for everything while implementing byzantine and ever-shifting safety regulations that each subcontractor must commit to following. Sure, it costs an order of magnitude more, but it's worth it for the severability.

Heavy machinery (in production) is designed that you cannot just walk into them and will stop, if you do. It is a bit different approach from software engeneering.

Folks who have grown up on the internet have likely seen plenty of video evidence to the contrary.

Well, I probably have to add, that my point of view comes from old school german engeneering (through my dad). His approach to building something vs. mine as a software dev are quite different ..

And the accidents that I know of, are where the security mechanism have been tampered with. A proper designed and certified german machine is hard to get injured by if followed protocol (and the reasons are past accidents and suing the manufactor).

Wow, this is absurdly callous IMO. How dumb does one have to be to deserve death? Why do we have lines on the road when discerning drivers could just avoid other cars without them? I think the idea that industrial deaths are inevitable/NBD/the fault of the victim is more outlandish than it’s implied to be in your comment

It’s victim-blaming which apparently is still a thing, even though we practically banned that talk in regards to rape victims.

Basically the same as when your company produced a pesticide that caused cancer and killed someone. There was no intent in the molecule, but there was at the very least negligence on the side of the producer; if they knew about any negative side effects but swept it under the rug, then I suppose it could qualify as manslaughter. In short, nothing really new here.

> If I create a machine that kills someone, my "intentionality" is fair game to be dissected in court, and ultimately I may be found guilty as if I had killed with my own hands.

The weapons industry (and its lobby organization, the NRA) would like to have a word with you...

weapons are made with intention to harm or kill and you using one wouldn't expect anything else, so this argument is weak at best.

Any regulation aimed at reducing the odds of accidental death has been fought tooth and nail by the industry and its lobbyists.

So not all that weak, really.

The argument is weak. The industry's argument isn't that. The industry's argument is money. Money is a very strong argument.

> Intentionality is a central concept in AI philosophy

Sure (and thanks for pointing this out) but this isn’t a philosophical piece, it was written for the Atlantic, and he’s being careless with his language.

Absolutely, but you're missing the most inflammatory part of all! The robot had to "eliminate the threat" -- when this was about a human who got in the way.

I agree with your first comment, but disagree on the second one. It is weird to say that the robot wanted to eliminate a threat, most likely it was simply not programmed to avoid obstacles given the environemnt it was in. He ascribes an intention that was not there. To be fair, it is a quote from a book and not the words chosen directly by the author, but he could have clarified.

The second one seems fair: he is describing the result of what the Tesla Autopilot is doing. I do not think that he means to say that the Autopilot is understanding that there is a red light but then it makes a consciuous decision to itgnore it. He is just saying that the robot is malfunctioning, as it should be clear from context:

  "Today’s AI-related robot deaths are no different from the robot accidents of the past. Those industrial robots malfunctioned, and human operators trying to assist were killed in unexpected ways. [..] Malfunctioning Teslas on Autopilot have deviated from their advertised capabilities by misreading road markings, suddenly veering into other cars or trees, crashing into well-marked service vehicles, or ignoring red lights, stop signs, and crosswalks".

I do not think there is a more neutral way of describing the events.

> most likely it was simply not programmed to avoid obstacles given the environment it was in.

Even more likely, it’s a manufacturing robot, with no vision at all except maybe for the pieces it is manipulating. It almost certainly has a line on the floor, or a railing even, to keep people out of its way.

In short, it has zero obstacle avoidance. I’d be interested in counterexamples.

> The second one seems fair: he is describing the result of what the Tesla Autopilot is doing. I do not think that he means to say that the Autopilot is understanding that there is a red light but then it makes a consciuous decision to itgnore it. He is just saying that the robot is malfunctioning, as it should be clear from context: "Today’s AI-related robot deaths are no different from the robot accidents of the past. Those industrial robots malfunctioned, and human operators trying to assist were killed in unexpected ways. [..] Malfunctioning Teslas on Autopilot have deviated from their advertised capabilities by misreading road markings, suddenly veering into other cars or trees, crashing into well-marked service vehicles, or ignoring red lights, stop signs, and crosswalks".

He describes Teslas as “no different” from non-intentional machines of the past, and then describes them with “intentional” language.

>I do not think there is a more neutral way of describing the events.

How about, “Industrial accidents happen through lack of human foresight and carelessness. The same is currently true of ‘AI’ products like Autopilot. But the time will come, and probably soon, where AI alignment and intentionality will be part of the conversation. We should prepare for that starting now.”

Not as catchy as “Robot rampage!” But more accurate and, as you say, neutral.

What if this author doesn’t care about intentionality? AFAICT you’re assuming they do because they use the active verb “ignore” for a machine, but that says little to me. We use language like this all the time for non-intentional systems: “the plant reached skyward”, “the economy soared”, etc etc etc.

I really really don’t think this is an article about convincing robots to be nice to us (AKA alignment); I think it’s an article about how we need stricter regulations for software that controls physical machines. Doesn’t that sound like a good idea?

The difference is that no one (very few people) would ever consider a plant to be sentient. So the metaphor works as intended. Since many (most?) people are willing to entertain the notion that AI can be intentional, the metaphor is ambiguous and fails.

I’m not arguing against the underlying point, I’m just disagreeing with how it’s presented.

To put it another way, we might as well call a steam train a robot who will murder any human who tries to obstruct it's desire to get to the next station, instead of jumping the track and going around.

Or a brick wall, a robot who will murder any human who runs into it fast enough.

That quote leapt out at me too. I think the reason it leapt out was because the rest of the article was so thoroughly objective, well reasoned, and rooted in historical precedence. Given the context in which it appeared, I’m willing to give it the benefit of the doubt as slightly expedient wording that was shorthand for the more nuanced understanding he clearly possesses.

Yeah this is a great example of what’s wrong with all our robots: their perception is garbage.

Reasoning is actually pretty good now. An AI can compose a sonnet on electron degeneracy pressure no-problem but ask it where the red ball is - and really inspect its localization of that red ball and it’s way off. Orders of magnitude of imprecision.

A modern AI can reason through the trolley problem and millions of variants. But it can’t see the person on the track with very high confidence, or be sure that the trolley is going to hit the person or not.

You're right about AI capabilities being much better in the "ungrounded" realm (like writing sonnets), but I would revise the statement a little to be more accurate.

AI systems are quite good at perception in a traditional sense (e.g. determining a red ball is in the sensor receptive field), but once actual cognition about how the red ball relates to the world around it becomes important, current approaches are limited.

I would argue that the first wave of perception is solved much more completely than the ability to reason about complex interactions of objects grounded in the physical world.

Disclaimer: I am a robotics research scientist

I think it matters how accurate something should be before we consider it "good". For many applications, a 99% success rate is unusably bad. A self driving car that recognizes 99% of red lights or stop signs is probably a criminal liability.

A red ball detector that operates at 60hz with a 99.9% accuracy will make an error on average once every 11.5 seconds. I think the issue is not so much the interactions of objects and the world but rather we're just wildly off-base regarding the base performance needed.

If a human driver isn't paying attention to the road, fails to see a stop sign, and drives past it causing an accident, isn't it a reasonable use of English to say they "ignored" the sign?

Ignoring can mean failing to notice something, as well as noticing but disregarding it. Words can often have multiple similar complementary meanings. I don't think the usage here is incorrect, or even misleading.

Any sort of ticket from the traffic police will say something like "failed to obey traffic control devices," which is more of a statement of fact. "Ignore" definitely has shades of nuance.

No it can’t. Every definition of ignore you will find has intention in it, words mean things.

You don’t think it’s important to draw a distinction between whether a robot was deliberately programmed to do something malicious as opposed to a bug which caused an oversight? Come on.

> Every definition of ignore you will find has intention in it

https://dictionary.cambridge.org/us/dictionary/english/ignor...

> to intentionally not listen or give attention to:

> to give no attention to something or someone:

Two definitions - one with intention, one without

> words mean things.

Yes, but they often mean multiple things. Very few dictionary entries only have a single definition for a word. You can't use "words mean things" to claim that "this word means only what I say it means, regardless of the other accepted uses it already has".

> You don’t think it’s important to draw a distinction between whether a robot was deliberately programmed to do something malicious as opposed to a bug which caused an oversight?

I do think there are times when it is very important to do that.

I just don't think that "ignore" is the right word to make that distinction, and isn't a distinction the authors were trying to make at that particular time, because if they were they would probably have used another word.

...at least, that's how I read it. It seemed clear to me. ¯\_(ツ)_/¯

I would say if the human driver was distracted and drove past a stop sign, he did not "ignore" the stop sign. He simply "missed" or did not see the stop sign. To ignore something, you have to be aware of what you are actively ignoring. Even in the link you provided for the "second" definition, look at the examples paired with it.

https://dictionary.cambridge.org/us/dictionary/english/ignor...

> Definition: to give no attention to something or someone:

> Example 1: They ignored our warnings.

> Example 2: The mayor ignored the hecklers and went on with her speech.

You should re-read those definitions. They both mean intentionally.

It's just a way for tech bros to dodge responsibility. The people who program AI should be liable for injury.

> > the robot simply determined that “the most efficient way to eliminate the threat was to push the worker into an adjacent machine.”

> This seems inflammatory and highly misleading. I could be wrong, but I highly doubt it: the robot didn't "determine" anything. It had assembly instructions to follow, and it followed them. The worker got in the way and was struck.

I was prepared to argue with you, but upon reading the article, I agree that the description you quote makes a huge category error.

I think it's believable that we'll soon get AIs that are advanced enough they can do agent modeling and that some subgoal-maximizer accident could happen where the robots determines "the most efficient way to accomplish <objective> is to take out that human so he doesn't turn me off", in which case the article's poetic description would be correct.

Except the quoted paragraph is talking about an industrial accident in 1981, so the whole "eliminate the threat" presentation is obviously absurd.