But what youre saying only layers over the fact that, to avoid getting caught and convicted, a sufficiently good explanation is all that's necessary.

Not quite. What I'm saying is that the explanation is valuable because it comes from the accountable party. That you have a good explanation does not absolve you of responsibility. The short and clear way to put it is this: You don't go to jail because YOU think you did something wrong, you go to jail because WE think you did something wrong.

If I'm a customer at a bank, and my loan has been denied, I don't care what some unaccountable AI system can come up with to explain that. I care what about how the accountable bankers justify putting that AI system into the process in the first place. How do they justify that AI system getting to make decisions that affect me and my life. I don't care about why the process does what it does, I care about why that is the process.

>If I'm a customer at a bank, and my loan has been denied, I don't care what some unaccountable AI system can come up with to explain that. I care what about how the accountable bankers justify putting that AI system into the process in the first place. How do they justify that AI system getting to make decisions that affect me and my life. I don't care about why the process does what it does, I care about why that is the process.

Well, a court/law just has to declare "AI" as allowed to be used in such decisions, and the whole recourse you describe vanishes though...

> my loan has been denied, I don't care what some unaccountable AI system can come up with to explain that.

> How do they justify that AI system getting to make decisions that affect me and my life.

so you, a priori, make the assumption that your loan _should've_ been accepted?

If the decision wasn't an AI, but some actuarial that calculates and computes based on a set of criteria, and the result is a denial, you could still make the same argument of "why is _this_ the process, instead of something else (that makes my loan acceptable)?".

> I care what about how the accountable bankers justify putting that AI system into the process in the first place.

Computers are already deciding. As to why: it's because it's their money they're lending.

I work in a bank and you're laughably wrong. Computers are at best assisting in the lending process but more realistically they're completely ignored. There is NO AI (at least recognizably, you could argue that there's some sort of very simple expert model embedded in the 10 credit limits) involved in the process at all.

Out lending process is based on the judgment of a few specific individuals, with more involved clients requiring approval from more senior people. All steps of that process can be overturned by the overseeing person, and that person is accountable for their decision.

I didn't say there was AI.

Article 22 of the GDPR states “The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her.”

Hidden in the definition of sufficient is "meeting the standards for absolution that an investigator would use". I honestly see no reason a system could not go so far as providing case law references or FICO + ECOA guidance as evidence for a compliant rejection, even if the underlying reason is some statistics based on sex.

And why would a perfectly reasonable bank tell its customers it's using AI? AI would provide the breadcrumbs and the loan officer would conduct a reasonable story using that - it's just parallel reconstruction at its finest. I imagine this is how credit scores work. A number comes out of the system and the officer has the messy job of explaining it.

I used to work in munitions export compliance and there intent really matters. It's the difference between a warning and going to federal prison. And intent is just a plausible story with evidence to back your decision, once you strip the emotion away.

More importantly, to the best of our knowledge, human thinking is not simply predicting the next word based on a past model.

but the fact that predicting the next words based on past models does such a good job of masquerading as human thinking indicates that much of human thinking isn't much more than that. It may even simplify the task of figuring human thinking by narrowing down the last details of of the smaller set of missing pieces.

an analagous result was obtained back when they mapped the small finite number of neurons in a snail brain, or the behavior of individuals in ant colonies. What looks like complex behavior turns out to be very simple under the hood.

for the vast ocean of the population who... not sure how to describe them... not good students when in school, would rather spend the bulk of their time with the TV blaring, eating cheetos and swiping on tik-tok, following the lives of celebrities and fighting about it, rather than do anything long term productive with their own lives... chat gpt may have already exceeded what they do with their cranial talents.

even a level up on the ladder, the types of office situations lampooned in The Office or Dilbert, are they doing much more as a percentage of time spent than chat GPT can do? "Mondays, amirite!?"

then the question becomes, are the intellectual elites among us doing that much more, or just doing much more of the same thing? I think a large portion of what we do is exactly what chap GPT does. The question is what is this other piece of our brains' that intervenes to say "hmm, need to think about this part a lot harder"

>such a good job of masquerading as human thinking indicates that much of human thinking isn't much more than that.

No, that doesn’t follow. It just means it roughly looks like human thinking.

Your comment is akin to saying a high resolution photo of a human has basically figured out a way to replicate humans. It looks like it in one aspect but it’s laughably wrong. Humans thought without language.

chatgpt is only laughably wrong to educated people. it's sadly better than most people can achieve on their own. Even for educated people, the laughable part is only a personal quirk that they focus on for psychological reasons, but taken seriously a paper written by chatgpt is a good place to start and would save an educated person a lot of time in just touching it up to make it presentable, or changing the query to pare down its search for a better starting point.

It's not doing nothing, it's doing a lot.

> taken seriously a paper written by chatgpt is a good place to start and would save an educated person a lot of time in just touching it up to make it presentable

If we ignore the minor requirement of the paper having any connection to reality, of course.

Judging from a lot of the commentary on this subject, it appears to me that a surprising number of people think that reality is little more than a quaint idea.

You’re still missing the forest for the trees. It is doing a lot and it’s better at producing text that is how a human would write about topics better than many humans.

That’s not at all related to being close to general human intelligence.

>but the fact that predicting the next words based on past models does such a good job of masquerading as human thinking indicates that much of human thinking isn't much more than that.

Going in a straight line does such a good job of predicting the next position of the car that it indicates driving isn't much more than going in a straight line.

Alonzo Church showed that lambda calculus can replace all of abstract numerical mathematics with manipulation of meaningless symbols laid out in straight lines, or curved around.

I would give this 10 points if I could! But that’s level 2 driving. It is not nothing.

I know that's meant to be snarky, but yeah. A good portion of driving is simply going where the car is already headed. A good portion of conversation is probably just stochastically stuffing filler words and noises into your ideas to aid in their reception.

Haven't you ever had a situation where you were speaking and you get distracted, but not interrupted, and your speech trails off or gets garbled after ten or so words? It feels sort of like you've got a few embeddings as a filter and you push words past them to speak, but if you lose focus on the filter the words get less meaningful.

I'm sure we're different than an LLM, but seeing how they generate words - not operate on meaning - rings true with how I feel when I don't apply continual feedback to my operating state.

I like the metaphore.

Politicians are exceptionally great at it, filling up conversations with nothing

Many would, I suspect, echo your thoughts. In terms of the people you're not sure how to describe (the 'vast ocean'), I'm just wondering how these ideas might find effect politically were they to be held by candidates. Clearly ascent to executive office would have to entail being economical with the truth (lying) to the electorate. Not that much of this does not happen already - some might think.

> but the fact that predicting the next words based on past models does such a good job of masquerading as human thinking indicates that much of human thinking isn't much more than that.

I'm not sure I agree with that logic. What it proves is that we as humans are bad at recognizing that text generation aren't thinking like we are... that doesn't necessarily mean thinking isn't much more than what it is doing though, it just means we are fooled. Given that nothing like this has existed before and our entire lives up until now have trained us to think something that looks like it is trying to communicate with us in this way is actually a human being I'd kind of expect us to be fooled.

Of course not, but it seems perfectly plausible that human brains are machines that take sensory input and memory and predict the next muscle stimulus.

Do we really know enough about human thinking to say that?

> human thinking is not simply predicting the next word based on a past model

Some evidence is emerging which indicates that the activations of a predictive system like GPT-2 can be mapped to human brain states (from fMRI) during language processing[1]. We seem to have at least _something_ in common with LLMs.

The same seems to be true for visual processing. Human brain states from fMRI can be mapped to latent space of systems like Stable Diffusion, effectively reading images from minds.[2]

[1] https://www.nature.com/articles/s41562-022-01516-2

[2] https://the-decoder.com/stable-diffusion-can-visualize-human...

One might argue that this comment is simply predicting the next word based on a past model.

We don't know that it's true that humans are "responsible" or "accountable" for their actions in any meaningful way. This smells like a "free will" argument.

IMHO, it's unlikely "free will" and "responsibility" are anything more than an illusion.

It really doesn’t matter for this discussion, because society and laws are structured as if people have free will. An AI has to live up to that as well.

Maybe I'm confused about the topic, then. What does "living up to that" mean here? Legal liability or something?

Yes, that was their point. Humans have accountability to one another - either legally or in less formal ways (such as being fired if you're making your co-workers uncomfortable).

Current machines simply don't have that kind of accountability. Even if we wanted to, we can't punish or ostracize ChatGPT when it lies to us, or makes us uncomfortable.

Their use & application can be made accountable.

The point of this thread is that the explanation a human gives carries some weight because the human know they may/will be held accountable for the truth of that explanation. ChatGPT's explanations carry no such weight, since ChatGPT itself has no concept of suffering consequences for its actions.

So, while both humans and ChatGPT can and do give bogus explanations for their actions, there are reasons to trust the humans' explanations more than ChatGPT's.

Whether or not we hold humans using ChatGPT accountable for their use of it is irrelevant to this thread.

Is the idea that we think the punishment factor makes the explanation somehow more reliable? Why would that be the case? It’s kind of a dark idea.

The only intent taking place here, is someone placing a process under automatic control. That's it. If I make a bot that posts random comments and it offends you, then I'm a bit responsible, but also you've gotten offended by a random comment.