Doesn't this also involve people not having another category aside from "cognition" to put natural language processing acts in? How many neural net constructors have a rigorously developed framework describing what "cognition" is?
I mean, there's a common counter argument to the "this is not cognition" position. That is: "you're just using 'cognition' as a placeholder for whatever these systems can't do". I don't think that counter-argument is true or characterizes the position well but it's important to frame one's position so it doesn't seem to be subject to this counter-argument.
Yes, of course this might be an even more primary reason; do not attribute to malice what can be explained by laziness. However, AI researchers should be wary of their language, that point is hammered in most curricula I have seen. So at the least it is negligence.
> I mean, there's a common counter argument to the "this is not cognition" position. That is: "you're just using 'cognition' as a placeholder for whatever these systems can't do".
Very valid point, but we know current deep learning mechanisms do not mimick human learning, language understanding and production in any way. They are far too simplified and specific for that.
Neural network activation functions are a far cry from neural spiking models and biological neural connectivity is far more complex than the networks used in deep learning. The attention mechanism that drives recent LLMs is also claimed to have some biological similarities, but upon closer inspection drawing strong analogies is not credible [1]. computer vs. human visual recognition tasks it falls apart and higher-level visual concepts. [2]
1. https://www.frontiersin.org/articles/10.3389/fncom.2020.0002...
(To be clear, I don't think this system is an AGI; just making the point that better goal posts are needed...)
The discussion on what constitutes intelligence and cognition is too vast and complex. The central point is that haphazard and unsubstantiated authromorphising language should be barred from scientific papers. We know that an LLM does not "know", "believe", "intent", or "feels" because they lack any form of integrated, general human-like intelligence. It is trivially correct to use different terms like "models", "predicts", "outputs emotive expressions" and avoid unevidenced claims of human behaviour.
I also wanted to nuance my opening post: I file the "don't humanise ML models"-comment of peer review under "minor issues". This means that it is an issue with the paper that does not prevent publication. This is not a hill I am willing to let good research die on.
Also, computation is not cognition. We need to focus on what distinguishes cognition from computation, if indeed these are distinct. I feel we are using the wrong words and thus having unproductive conversations regarding this topic (in general).
The CS people seem all too happy to humanise computation, probably because they had less direct teaching regarding the cognitive mechanisms behind cognition and language production.
I name my bots and machines and of course in daily discussions the loaded words ("thinking", ""believing", "meaning", "knowing") are used. Simulating any human behaviour will elicit a sympathetic response, especially if it has utility to the user. But in the context of peer-review of scientific engineering papers that is inappropiate.
For the journal paper genre which is highly technical, more specific and appropriate terminology is always available that sidestep the whole issue.
It doesn't mean that there is "something there" with LM tho - just that we're good at tricking ourselves to think that way.
I do agree that we can't ascribe cognition to machine learning.
But I also believe that we can't ascribe that it's NOT cognition. Why? Because we don't even truly understand what "Knowing" or cognition is. We can't even ascribe a quantitative similarity metric.
What we are seeing is that those inputs and outputs look remarkably similar to the real thing. How similar it is internally is not a known thing.
That's why even though you're an NLP researcher, I still say your argument here is just as niave as the person who claims these things are sentient. You simply don't know. No one does.
So basic in fact, I was thought this in elementary school. So far ad-hominem attributions of naivety.
Anyone that humanises computation is not only committing an A.I. faux-pas but are going against the basic scientific method.
Yes you're correct. So you can't make the claim that it's NOT cognition. That is my point. You also can't make the claim that it is cognition which was the OTHER point. Completely agree with your statement here.
But it goes further then this, and your statement shows YOU don't understand science.
>So basic in fact, I was thought this in elementary school. So far ad-hominem attributions of naivety.
No science is complex and basically most people don't understand the scientific method and it's limitations. It's not basic at all, not even people who graduate from four year colleges in STEM fully understand the true nature of science. Or even many scientists!
In science and therefore reality as we know it; nothing can be proven. This is because every subsequent observation can completely contradict an initial claim. Proof is the domain of logic and math, it doesn't exist in reality. Things can be disproven but nothing can actually be proven. That is science.
This is subtle stuff, but it's legit. I'll quote Einstein if you don't believe me:
"No amount of experimentation can ever prove me right; a single experiment can prove me wrong." - Einstein
And a link for further investigation: https://en.wikipedia.org/wiki/Falsifiability
Anyway all of this says that NO claim can be made about anything unless it's disproof. Which is exactly inline with what I'm saying.
Still claims are made all the time anyway in academia and the majority of these claims aren't technically scientific. This occurs because we can't practically operate on anything in reality if we can't in actuality claim things are true. So we do it anyway despite lack of any form of actual proof.
>Anyone that humanises computation is not only committing an A.I. faux-pas but are going against the basic scientific method.
But so is dismissing any similarity to humans. You can't technically say it's wrong or right. Especially when the outputs and inputs to these models are very similar to what humans would say.
This is basic preschool stuff I knew this when I was a baby! I thought everybody knew this! <Joking>.
Machine cognition is a similarly extraordinary claim that’s going to need a lot more evidence than a just-right sequence of inputs and outputs.
Also, you can simply ask ChatGPT “A story about pyramids of Giza being built by aliens”, and it comes up with a reasonable story. This stuff is scary.
I have already incorporated into my daily use (as a programmer). It has huge flaws but the output is anecdotally amazing enough that the claim of "cognition" is not as extraordinary as you think it is.
Especially given the fact that we don't even fully understand what cognition is, the claim that it is NOT cognition is equally just as crazy.
You seem to agree with me even though your interpretation of falsifiability is inverted: I am not asking that authors make a claim that their models do not mimick human intelligence. Like OP, I ask them that they do not make that positive claim, i.e. omit humanising language unless they can substantiate it with evidence.
Also agreeing that current LLMs are probably not sentient in any meaningful way. But what I don't like about the discussion is is that it steers in a direction where it would fundamentally be seen as bad science to claim that any AI model could be conscious - and I don't see that covered by the scientific method either, for the reasons that onetwoonetwo explained.
This reminds me a bit of the discussion whether or not animals can be conscious/experience emotions/feel pain, etc.
The input to chatGPT is a textual interface, the output is letters on a screen. That is the exact same interface as if I were chatting with a human over a chat app.
Your getting into the technicalities of intermediary inputs and outputs. Well sure... analog data seen by the nueral wetware of human brains IS obviously different from the textual digital data inputted into the ML model. There are very different filters and mechanisms at work here. For sure.
HOWEVER, we are looking for an isomorphism here. Similar to how a emulated playstation on a computer is very different then a physical playstation... an internal isomorphism STILL exists between hardware and the software emulating the hardware.
We do not know if such an isomorphism exists between chatGPT and the human brain. This isomorphism is basically the crystallized essence of what cognition is if we could define it. If one does exists it's not perfect... there are missing things. But it is niave to say that some form isomorphism isn't there AT ALL. It also niave to say that there is FOR SURE an isomorphism.
The most rational and scientific thing at this point is to speculate. Maybe what chatGPT is, is something vaguely isomorphic to cognition. Keyword: maybe.
It is NOT an unreasonable speculation GIVEN what we KNOW and DON'T KNOW.
This in itself is a claim made without evidence. Which is my point. The claim as it stands cannot be made either way. We simply don't know.
Using humanising language is equivalent to attributing human-like cognition to ML models. Unless there is very strong evidence that there are analogies between the specific modeling mechanism and human-like intelligence, it is always incorrect to positively assert these claims without evidence. In science, you can only assert that for which there is evidence, strong claims like the above require strong evidence.
Your criticism should be calling on them to post more respectfully and humbly, rather than shunning them. We don't know if they're a scientist (let alone that they never will be) and shouldn't assert they aren't, and everyone is entitled to post from throwaway accounts. Everyone should be welcome to question assertions, even nonexperts (though nonexperts should acknowledge their limitations & refrain from disrespectful language).
What is ironic is you hiding behind throwaway accounts and asserting superior knowledge while fundamentally misunderstanding the issue of positive claims in science.
Read the conversation. You are accusing me of something vile which the GP started. GP Literally said that science was something elementary school level shit implying that I was so damn stupid I was uneducated. You're only taking his side because You agree with him, you are not seeing who drew first blood and you are burning the witch because it serves your own agenda. He insulted me DELIBERATELY. I'm also not technically insulting him. I don't think he knew what I told him.
>You fundamentally misunderstand the scientific method which includes falsifiability but is not limited to it
This is fundamentally incorrect. You are just like the GP. You lack of knowledge. Did you not see the quote from Einstein? My statements are legitimate.
The highest most definitive statement one can make in science is falsification. The next highest form of statement is causality and the next is correlative and everything after that is just informal qualitative stuff. That's it. After falsification everything becomes dependent on the sample and all statements are open to being proved completely wrong at any point in time. Causative and correlative statements are never definitive. But falsification is definitive.
Let me repeat. The only definitive claims science can make about reality is falsification. That's it.
There is no form of provable claims in science. None. If a scientist makes such a claim in a scientific paper, it is not technically science. It is simply a speculative claim on an academic paper. This is not philosophy this is science in it's most fundamental form.
>What is ironic is you hiding behind throwaway accounts
Hiding behind throw away accounts? Are you joking? What is there to hide that everyone else is hiding here? Are you implying you're not hiding, why don't you tell me your real full name if your not hiding as well? The only reason why I made a new account is because I wanted to comment here. That's it. Are you demanding I put my name here?
>moved the goalpost away from peer review for journals
Let me quote what was said in GP's FIRST sentence and FIRST post.
"I am NLP researcher who volunteers for peer review often and the anthropomorphisms in papers are indeed very common and very wrong. "
There are TWO GOAL POSTS listed here. anthropomorphisms LISTED in papers and anthropomorphisms being "VERY WRONG." I did not move a goal post. I am simply refuting a goal post that the GP listed himself and that YOU missed.
>while fundamentally misunderstanding the issue of positive claims in science.
Positivism in science is largely a philosophy. It's abstract enough such that it's not worth talking about. This is actual philosophy. Falsification is fundamental enough that it is intrinsic to understanding the scientific method. It's not "just" a philosophy any more then logic or math is a "philosophy". Literally. If you don't get that the entire scientific method and reality in itself is ONLY subject to falsification, you then you don't understand science.
I'll repeat Einsteins quote:
"No amount of experimentation can ever prove me right; a single experiment can prove me wrong."
Literally the man is saying that there is no science in the universe that can prove any of his theories right. If you don't understand the meaning behind why he said this, again, you don't understand science. This is fundamental it is basic. Not fundamental in the sense that everyone should understand it, but fundamental in terms of foundational to reality, to science itself.
Extremely petty, insubstantial and toxic, you have nothing insightful to add.
People do this because mirroring cognition to machine learning lends credence that their specific modeling mechanism mimicks human understanding and so is closer "to the real thing". Obviously this is almost never the case, unless they explicitly use biomimetic methods in which case they are often outperformed by non-biomimetic state-of-the-art approaches.
Thanks OP for giving me citation ammo to refer to in my obligatory "don't humanise AI" section of reviews. (It is so common I copy paste this section from a template).