News for hackers. Stuff that matters.
Re:We come to bury ChatGPT, not to praise it by tambourine_man on Monday February 06, 2023 @10:11PM (#34688226)
Please provide evidence that intelligent speech isn’t simply predicting the next word, as it’s obviously not self-evident to me.
Re:We come to bury ChatGPT, not to praise it by lukev on Monday February 06, 2023 @10:46PM (#34688501)
I'm not sure it counts as "evidence" because I can't show it to you, but based on personal introspection, whenever I write something it starts with a barely-linguistic thought I want to communicate that gradually gets expanded into as much prose as is necessary to get across the concept whilst establishing context and defending from the most probable detractors.

I literally never just start writing and then just keep adding words.

Re:We come to bury ChatGPT, not to praise it by tambourine_man on Monday February 06, 2023 @11:00PM (#34688609)
It’s not evidence, sure, but what you describe, the “feel of it” is most of what we have to work with at the moment, unfortunately.

But it clearly is not self-evident that intelligence is or isn’t prediction.

Nothing is self-evident in a field we know so very little about. That’s my main objection, we should be humble and curious, not dogmatic.

Re:We come to bury ChatGPT, not to praise it by orwin on Tuesday February 07, 2023 @04:27AM (#34690569)
It's pretty much self-evident when you worked with 3-6 that this isn't how it work. The idea come before the vocalization.

Or even when you talk in a foreign language you're semi-competent with.

Re:We come to bury ChatGPT, not to praise it by tambourine_man on Tuesday February 07, 2023 @05:31AM (#34691018)
I’ve spent many years with infants and learned 3 languages aside from my mother tongue. I don’t find it self-evident at all.

You’re gonna need to elaborate why you think so if you’re willing to convince someone.

Re:We come to bury ChatGPT, not to praise it by orwin on Tuesday February 07, 2023 @06:42AM (#34691466)
That the idea form before it's vocalized?

A three year old reaching toward any toy, then saying 'mine!'.

Stumbling on a specific word, stopping your sentence, then proceed to replace it with a combination of words.

Or any time you paraphrase basically.

Re:We come to bury ChatGPT, not to praise it by tambourine_man on Tuesday February 07, 2023 @07:29AM (#34691856)
Not true in my experience. To me, words come first, the idea is a consequence I realize after a few of them have been stringed together.

Knowing where the words come from, how “the brain” knows the next one before my conscious self does has always been a mystery to me. I’ve heard Sam Harris describing a similar feeling in his podcast.

Re:We come to bury ChatGPT, not to praise it by orwin on Tuesday February 07, 2023 @08:33AM (#34692490)
You never paraphrased something despite knowing the exact word?

[edit] Or explained high level CS concepts to an undergrad at your school?

Re:We come to bury ChatGPT, not to praise it by tambourine_man on Tuesday February 07, 2023 @09:08AM (#34692882)
Sure, but how's that indicative that prediction isn't taking place?
Re:We come to bury ChatGPT, not to praise it by orwin on Tuesday February 07, 2023 @05:53PM (#34701049)
I have the idea before i have thought of the word i would use to express it. Sometimes, i don't even have word to express it, "you know what i mean?".

And like i said. Children will often act, then talk. Or seek their words for long period of time, and when you propose a word to help them vocalize, they can accept it or reject it.

I have another example. Do you ever have slip of the tongue? You're talking about "yoga", tell yourself while speaking "i really shouldn't say 'yoda' here", and say the word "yoda" anyway? This happened to me a lot when i was younger (my speed of thought is lower those days, i don't think i slipped this way in two or three years). I still think a bit faster than i speak tbh.

Re:We come to bury ChatGPT, not to praise it by tambourine_man on Tuesday February 07, 2023 @07:52PM (#34702424)
All of that happens to me as well. I just don’t see how it couldn’t be explained by prediction from probability distribution.

The fact that you can’t find a word could be that you’re stuck in a Markov chain wall or loop:

https://news.ycombinator.com/item?id=34688673

The thing is, we just don’t know how any of that work. Our speculation is as good as any.

Re:We come to bury ChatGPT, not to praise it by phaedrus on Monday February 06, 2023 @10:48PM (#34688515)
I've always hated the idea promoted by some AI researchers that "intelligence is just prediction." Specifically in the context of your challenge, since a brain must decide the next word before uttering it, is it not vacuously true that any conceivable method of producing speech is "predicting" it? You're asking for evidence that it isn't; I'm asking for evidence that question has any meaning.
Re:We come to bury ChatGPT, not to praise it by tambourine_man on Monday February 06, 2023 @11:04PM (#34688632)
Sure, any production in that case can be classified as prediction. What I’m saying is that such distinction, if at all possible, is far from self-evident.
Re:We come to bury ChatGPT, not to praise it by breckenedge on Monday February 06, 2023 @10:55PM (#34688570)
Anytime you say “uh, what’s that word again?”

So you’re thinking of a concept, you can picture it and it’s impact on your environment, but can’t quite remember the word for it.

Re:We come to bury ChatGPT, not to praise it by tambourine_man on Monday February 06, 2023 @11:13PM (#34688673)
Or, you reached a point in the Markov chain where none of the next guesses rank high enough if the whole paragraph is taken into account. It often feels like that, in fact, you can’t figure it out until you take another “route”.

Of course, my anecdote is just as valid as yours. My point is we have no freaking idea what’s going on and shouldn’t jump to conclusions.

Re:We come to bury ChatGPT, not to praise it by root_axis on Monday February 06, 2023 @11:40PM (#34688848)
The content of intelligent speech is not driven by statistical relationships between the spoken lexemes, this is self-evident.

For example, let's say you're planning to approach your boss about receiving a raise, the contents of your speech will be driven by a personal theory of mind regarding how your boss perceives you, your own memory of who you are and how you like to be perceived, the cadence and tenor of speech you imagine your boss is receptive to, information contextually relevant to the purpose of the speech like your proven history of success within the organization, as well as a hundred other variables driven by physiological factors, culture, social habituation and more. "Predict the next word" just doesn't describe intelligent speech in any meaningful way.

Re:We come to bury ChatGPT, not to praise it by tambourine_man on Tuesday February 07, 2023 @05:48AM (#34691122)
So, you thrown in theory of mind, physiological factors and culture without defining what that means or showing that they don't all derive from the same underlying statistical relationships in order to make your argument more robust? That's not how it works.

Defining intelligence is fundamental philosophical problem, a discussion that's thousands of years old in the making. To call something in this field self-evident is either ignorance or self denial.

Re:We come to bury ChatGPT, not to praise it by tarsinge on Tuesday February 07, 2023 @02:09AM (#34689681)
Someone deaf and blind still form thoughts but don’t output them in words. Words follow cognition, it’s sometimes obvious when you have a clear concept in mind but forget the word.

Edit: the thought is maybe a prediction, but language seems not to be the abstraction it’s operated on.

Re:We come to bury ChatGPT, not to praise it by tambourine_man on Tuesday February 07, 2023 @05:33AM (#34691029)
Of course they do. They still have language. How else would they interact with the world?
Re:We come to bury ChatGPT, not to praise it by pnt12 on Tuesday February 07, 2023 @08:25AM (#34692402)
You're asking for the evidence of a non existing factor in a complex process not fully understood... I'd say it's on you to provide it.

Why is predicting the next word proof of intelligent speech?

Re:We come to bury ChatGPT, not to praise it by tambourine_man on Tuesday February 07, 2023 @01:08PM (#34696496)
That’s not what I’m asking.

The grandparent said it’s self-evident that humans do not simply predict the next word. I asked for evidence of that self-evidence.