One of them is that we only have language with which to build an AI - language and symbols. Yet these nevertheless capture absolutely nothing about what it is to perceive and understand, on a human level, so yeah. It’s kind of what Searle was getting at - symbolic outputs and translation based on rules doesn’t get you intelligence, even if it produces a system that looks as if it is intelligent.
There’s also that we have an intersubjective attunement to others that is pre-linguistic. Ultimately “meaning is use” which is a pithy take on later Wittgenstein- there’s nothing in language which means anything at all. So it’s not that it wouldn’t be able to use language or concepts better than we do, it’s that we change with the AI and don’t accept them - particularly as they start to become uncanny or unpredictable.
There really is something to sentience that we all know that cannot be reduced to language or machinery.
Our language about machinery, and intelligent machinery, is full of usage mistakes and categorical errors that we overlook, but which are more serious when analysed.
We are complete beings - my intelligence is part of a system, and it makes no sense to abstract it out of that system in order to nail it down.
“Pain” cannot be reduced down to or captured in language. Suffering is part of our intelligence and we’d be nowhere without it.
So ideas kind of like that and more.
AGI would have to suffer to become general enough to warrant the use of the word general.
Jo Cameron doesn't have general intelligence? This seems absurd. Intelligence is orthogonal to phenomenology and affective states. People aren't worried about AGI because it might have a Cartesian theater, the worry is that it might be more competent than humans and put humans out of a job. The semantics of whether it "truly" has intelligence is irrelevant.
>Suffering is part of our intelligence and we’d be nowhere without it.
It might be part of our intelligence, but why would it need to be part of machine's intelligence? GPT4 is already beating humans at theory of mind tasks[1], and I doubt that it suffers. Our suffering is an evolutionary stroke of bad luck. It has nothing to do with intelligence itself, and it would have been better if we had evolved some other way that didn't need it.
I’d not heard of Jo Cameron but one outlier, for me, wouldn’t hyper-negate the whole idea out of existence. A huge amount of what I suffer with is not physical pain.
It's not just physical suffering that she's supposedly immune to. She's also immune to all psychological suffering as well. And she is married, has kids, and is perfectly healthy. If she's in fact the real deal (I don't know this myself, but I haven't seen anyone debunking her) then dismissing her as an outlier would be a remarkable form of complacency. If the effect can be understood and replicated, suffering will become fully as needless as it deserves to be, with its final eradication mourned probably as much as the death of smallpox.
> If LLMs are beating humans in theory of mind tasks, your theory of mind is incorrect.
Or, LLMs have better theory of mind than most humans do, which is the finding of that study. Is it metaphysically impossible? If your mental image of how LLM cognition works is the same type of expert system that Searle was writing about back in the day, it's refreshingly bizarre to read how LLMs appear to work:
https://www.astralcodexten.com/p/god-help-us-lets-try-to-und...
Where is the Chinese Room in this? I don't see one. Just a lot of complex and vague conceptual associations mediated through neural connections. Whether or not these models are conscious or have an inner life, they seem to be doing just fine understanding concepts.
>We are complete beings - my intelligence is part of a system, and it makes no sense to abstract it out of that system in order to nail it down.
The more I read rebuttals of these kinds, the more I think AGI believers might be right.
Can you elaborate on why you believe AGI is nonsense?
> ignoring the way more value-generating aspects (synthesis, categorization, summarization, etc.)
Summarization is one of the most common applications of LLMs I see. I expect if I knew more precisely what you meant by the other two we'd find examples of those, too. Everybody is trying to find out what these tools can do and throwing everything at the wall to see what sticks. People are doing exactly what one would expect them to do.
John Searle is much smarter than I am and he elaborated plenty on this in the 80s. (But more saliently, I think it's absurd to think our brains, or brains in general, do polynomial curve fitting in any meaningful sense.)
> Summarization is one of the most common applications of LLMs I see.
This is true, though the workflow is extremely cumbersome.
> People are doing exactly what one would expect them to do.
I don't disagree, I think people are moving in the right direction (albeit slowly). Maybe the generative aspect is just more awe-inducing rather than "oh hey this thing creates calendar events out of any date," even though the latter would be way more useful.
The argument that any non-materialist position (panpsychism, substance dualism, monadism, etc.) must "believe in ghosts" is comically reductive. Metaphysics is a thing, you know. So is moral philosophy. I mean, heck, so is logic. In all these subfields, arguing that their respective first principles are purely materialist is an uphill battle. I get a feeling that post-2000s materialism is basically logical positivism 2.0 (and we know how that ended).
> It's not a claim that AGI is nonsense in principle
You're playing a semantic game here; but in any case, understanding is a function of intelligence, so (working backwards), you're actually shooting yourself in the foot, anyway.
"Ghost in the machine" is from Ryle. It's quite apropos here.
> You're playing a semantic game here; but in any case, understanding is a function of intelligence, so (working backwards), you're actually shooting yourself in the foot, anyway.
I honestly don't know what point you're trying to make.
To recap, you said that:
1. You don't believe in this AGI nonsense.
2. Searle elaborated on why AGI is nonsense in the 80s.
And I am clarifying for you that:
1. Searle did not elaborate on why AGI is nonsense in the 80s. He doesn't claim that AGI is nonsense at all.
2. His arguments are specific to the methods that were called AI, and a bunch of technical and philosophical claims about those methods and about the human mind, at the time he was writing. Ditto for Dreyfus.
What you can get Searle to commit to is the idea that human-like AGI can't be a program executed on computers as we think of them today. He's an ultra-materialist, like Ryle, and thinks the physical reality of brains is essential to the minds they produce. To make a mind, you need to make a brain. (In other words, he thinks "Lena" is nonsense: https://qntm.org/mmacevedo.)
So he would say that ChatGPT is no closer to being human-like AGI than ELIZA was, since it's no closer to having something like a human brain, and if all you meant by "the whole AGI nonsense" is that ChatGPT isn't a step toward human-like AGI, then he'd agree. But he wouldn't say that human-like AGI itself is nonsense, precisely because he doesn't believe in ghosts.
And yes, by "AGI nonsense" I meant that "ChatGPT isn't a step toward human-like AGI" since the main chatter these days is about how ChatGPT/Claude/etc. is the harbinger of AGI. I think Searle's argument is particularly strong because you don't even have to believe in something "spooky" happening in our heads. In fact, I'd probably extend "AGI nonsense" to "AGI is flat out not possible" but that's more of a hunch and the full argument invokes things like the non-computability of various physical phenomena.
Thanks for chiming in as I'm completely unfamiliar with Searle.
I don't believe in immaterial human souls, therefore self-aware AGI appears to be a near inevitability of technological advancement from my perspective. To suggest that it's impossible is, in my opinion, absurd.
Why would an AI need to use the same mechanisms as a human to think? Most machines we create are quite different in character from their natural counterparts (what bird has a propeller?)
>and as a trained philosopher, I don't buy into the whole AGI nonsense
It's hard to imagine Socrates using this form of argument.
> To die proudly when it is no longer possible to live proudly. Death of one's own free choice, death at the proper time, with a clear head and with joyfulness, consummated in the midst of children and witnesses: so that an actual leave-taking is possible while he who is leaving is still there
(Twilight of the Idols, 1888)
That really is Nietzsche’s angle on it if you care to go looking rather than cite him at me to disprove what I said. I think it’s in Will to Power but it’s been a while. You’ll also notice throughout that Nietzsche isn’t a dialectical thinker. He doesn’t go round in circles trying to find the antithesis of his polemics – he’s calling it how it is for him.
Edit: some of his perspective on Socrates is in Twilight - Google: Nietzsche Socrates Buffoon.
I'm actually quite bearish on AI on the very long term (and as a trained philosopher, I don't buy into the whole AGI nonsense), but I do think AI will redefine how we interact with our computers/devices in the short-to-medium term. These days, everyone's too enamored with the generative aspect (which is, for the most park, bunk), and for whatever reason ignoring the way more value-generating aspects (synthesis, categorization, summarization, validation/invocation, etc.).