I suspect he posited trust in juxtaposition to reliability, rather than veracity.

More importantly humans have ways to detect deception from other humans, be it through body language or other cues. With only text it is very hard to determine whether the model is lying to you or not.

Even in text, there is more context. For example, I am more likely to trust the wikipedia article about a deeply technical topic than an article about politics or a celebrity, because the technical article is far more likely to only be edited by people who are actually very knowledgeable on the topic, and there is very little incentive to lie (in general, there are exceptions).

The other difference is that over time we build up a network of people we consider to be knowledgeable and honest. Current LLMs can never match that because their output is controlled guessing.

> A common response to this by AI advocates is to point out that humans lie all the time, as long as the AI lies less than humans (debatable at this current point anyway) its an improvement.

This is also Elon Musk's justification for self-driving cars: "They make fewer mistakes than humans and are therefore safer."

It's true that self-driving cars avoid many of the mistakes of human drivers, but they also invent whole new categories of fatal mistakes that humans rarely make. And that's why Musk's argument is garbage.

If the goal is to reduce the number of fatal mistakes, why is that argument garbage?

Because it's unacceptable to replace a perfectly good driver in control of their vehicle with a vehicle that might just randomly kill them.

Traffic accidents don't happen randomly at all. If you are not too tired, drunk or using any substances, and not speeding, your chances of causing a serious traffic accident are miniscule.

These are all things you can control (one way or another). You can also adjust your driving to how you are feeling (eg take extra looks around you when you are a bit tired).

This feels like the trolley problem applied at scale. Will you deploy a self driving system that is perfect and stops all fatal accidents but kills one randomly selected person everyday?

Nope: there is no moral justification to potentially kill a person not participating in the risky activity of driving just so we could have other people be driven around.

Would you sign up for such a system if you can volunteer to participate in it, with now that random killings being restricted to those who've signed up for it, including you?

In all traffic accidents, there is some irresponsibility that led to one event or the other, other than natural disasters that couldn't be predicted. A human or ten is always to blame.

Not to mention that the problems are hardly equivalent. For instance, a perfect system designed to stop all accidents would likely have crawled to a stop: stationary vehicles have pretty low chances of accidents. I can't think of anyone who would vote to increase their chances of dying without any say in it, and especially not as some computer-generated lottery.

> Would you sign up for such a system if you can volunteer to participate in it, with now those random killings being restricted to those who've signed up for it, including you?

I mean, we already have. You volunteer to participate in a system where ~40k people die in the US every year by engaging in travel on public roadways. If self-driving reduces that to 10k, that's a win. You're not really making any sense.

But none of that is random.

Eg. NYC (population estimate 8.3M) had 273 fatalities in 2021 (easy to find full year numbers for): https://www.triallaw1.com/data-shows-2021-was-the-deadliest-...

USA (population estimate 335M) had 42,915 (estimated) according to https://www.nhtsa.gov/press-releases/early-estimate-2021-tra...

USA-wide rate is 1 in 7,800 people dying in traffic accidents yearly, whereas NYC has a rate of 1 in 30,000. I am sure it's even lower for subway riders vs drivers. Even drivers, somebody doing 4k miles a year has different chances than somebody doing 40k. People usually adapt their driving style after having kids which also reduces the chances of them being in a collision.

Basically, your life choices and circumstances influence your chances of dying in a traffic accident.

At the extreme, you can go live on a mountaintop, produce your own food and not have to get in contact with a vehicle at all (and some cultures even do).

FWIW, I responded to a rethorical question about killings being random: they are not random today, even if there is a random element to them!

If you want to sign up to a completely random and expected chance of death that you can't influence at all, good luck! I don't.

In traffic incidents, humans drivers are rarely held accountable. It is notoriously difficult to get a conviction for vehicular manslaughter. It is almost always ruled an accident, and insurance pays rather than the human at fault.

Traffic fatalities often kill others, not just the car occupants. Thus, if a self-driving system causes half as many fatalities as a human, shouldn't the moral imperative be to increase self-driving and eventually ban human driving?

> If you are not too tired, drunk or using any substances, and not speeding, your chances of causing a serious traffic accident are miniscule.

You realize that like.. other people exist, right?

You realize that I said "causing"?

For people to die in a traffic accident, there needs to be a traffic accident. They are usually caused by impaired humans, which means that they are very often involved in traffic accidents (basically, almost all of them have at least one party of the sort), whereas non-impaired people mostly do not participate in traffic accidents as often.

This is a discussion of chances and probabilities: not being impaired significantly reduces your chance of being in a traffic accident since being impaired significantly increases it. I am not sure what's unclear about that?

I don't even think they make less mistakes than humans period: they usually compare numbers against all driving instances including those performed by incapacitated humans (drunk or extremely tired human drivers make the bulk of the "mistakes", but humans can—somewhat—control whether they do any driving then).

> If your compiler lied about your code being valid, I doubt the general response would be "meh, its only done that once, I've lied far more than that"

Any language with an unsound type system will do this occasionally. This probably includes a majority of all code being written today: C, Java, and Typescript are all unsound.

> A common response to this by AI advocates is to point out that humans lie all the time

That’s true. But what do people do when someone lies frequently? That’s right, we stop trusting them.