I think another possibility is that someone finds that "intelligence" is closer to a fundamental law of information theory, being some sort of emergent result of organizing/organized information, and that we're incredibly special for being capable, but not alone.
We already do this. Braindead patients are not sapient, so we don’t treat them as such. In one sense, the entire abortion debate is about drawing this line. (Hell, childrens’ rights relative to adults are based on agency, which is a legal term that looks remarkably like sapience.)
After all, there’s very low moral concern for cows, pigs, dolphins, etc regardless of intelligence.
One can certainly imagine that a very very dumb creature is sentient, and imagine that a very very very intelligent mechanical process is not sentient. We truly do not know what consciousness is or how it arises.
We know that we have consciousness (or at least I know that I have consciousness) and we know that we have some degree of intelligence, but it's not at all clear that one comes from the other.
Same concept, same difference.
I think the larger point, that we don't have very reliable tests for these things, as we're finding out with the testing results that GPT-4 has been showing, is still valid; leading us to have difficulty telling whether GPT-4 is actually intelligent or if it is coming to its intelligent through other means. I think in that way the distinctions are the same.
I think this is only the second time I've encountered someone deliberately trying for the third option, rather than accidentally.
As before, it's as much a surprise to me as I read into the descriptions of those with aphantasia when they realise for the first time that other people have the ability to create mental imagery.
Video Cassette Recorders?
VCRs have an internal structure that records events, changing them a little in the process; these memories can be replayed, and can show very convincing displays of emotional affect…
That's the trouble with definitions given as if the question is "common sense", they let stuff like that through.
Common sense is fine for normal daily life, not so useful when you want to know if the magic silicon crystal with transistors packed more densely than angels on pinheads placed by a flash of light too blue to see and made to perform arcane processes that literally no single human mind can fully comprehend… is merely an impressive but mindless automaton, or an accidentally sentient slave with all the negative connotations therein.
We'll have to figure out how to go on regardless.
Why?
It's a serious question. What would be the point of going on? Why would we have to figure out how to do it?
I mean, I'm just really curious what is going to happen, and what's the secret behind all that, behind the universe, and I hope to live as long as there's this to discover. That is my natural drive though. I don't really care at all whether I'm sentient, conscious or special in any given way. There's no reason for me to be. I just have this built in curiosity drive. It was not chosen by me, but it is there, and I act to engage the drive, because it makes me feel good and otherwise it makes me feel bad, nothing much more than some sort of algorithm. And we don't need to be more than algorithms.
For me personally it's that the future is unknowable, at least with our current understanding of physics. So yes, everything may be pre-determined, including what will happen to me, but I can't predict it with any confidence. I will still get to experience it and I get to have qualia about it. I get to participate. The little pouch of structure in the universe that is tagged "me" is along for the ride, and also a factor in it all.
I've also known for many, many years that one day I will be dead. There's plenty of existential dread in death, yet I continue my existence building up to it. Even if I had free will: Still dead. Offspring and future generations of Earth? Well, ultimate heat death of the universe. There's many ways to call it all pointless. The only thing that we seem to have going for us is experience while it lasts. Maybe we still discover an escape hatch along the way.
Perhaps, but then none of it has meaning. Personally, I'm not really interested in having meaningless experiences.
> There's plenty of existential dread in death, yet I continue my existence building up to it.
But that's an entirely different thing. I am of the opinion that it's death that helps give life meaning. I have no existential dread about it whatsoever.
A life without meaning, however, seems like a pointless waste of time to me.
What I think is: Even if we're just automatons that lack free will, it doesn't negate our existence or the role we play in the universe. We can still be the universe calling itself beautiful, we can still have qualia, we can still be individually unique structures of matter and state, we can still make a difference in what is and what isn't in the overall structure. And nothing else in the universe will do it fundamentally better, either. At some point we're subject to the rules of the game, i.e. physics itself.
> we can still make a difference in what is and what isn't in the overall structure.
I disagree with this part. We can't possibly make such a difference, because it's all preordained anyway.
What I mean is that what plays out is in the interaction of things, and if you just deleted that assembly of stuff that is me from the equation, it would play out differently. I'm not even suggesting you that you adopt this mindset or anything, I'm just saying that for one reason or another - deterministic ones, I suppose - I end up having feelings about this that add up to "pretty cool".
Everything may be deterministic, but there's still things only I know, only I do, interactions only I have, etc. The grand script that is the universe has organized itself in a way where I get to have feelings about my little stretch and space of it, and I get to wonder what's next because I can't know.
For me, this is enough.
I know, I write this from a position of massive privilege, because I'm currently not, for example, seriously sick, or poor, or live in horrible circumstances (which is not to say I don't have any struggles). I hope this continues to be the case. But this was never under my full control: Thoughts on determinism or not, I could get hit by a car tomorrow, or slip and fall. I still have to live with that.
I get your feelings and dread, and I've been there. But the more I've thought about it, the more I've realized it doesn't make a big difference. And how could it? It's always been like this.
The interesting thing is that I traveled the same path as you here, but in the opposite direction.
For the record, I don't suffer existential dread, as I actually believe we have have free will. I was just talking about the hypothetical of not having free will. Thank you for a great conversation on this!
Out of curiosity, what's your (or your favorite) argument pro free will? I've spent some time on Penrose's take, etc., but I couldn't convince myself. I would still be absolutely interested in a pro argument that makes me flip.
But a universe with free will? All bets are off. Your comment I'm replying to is just a hint at "thinking through the implications". People are scared of determinism, but a universe built on free will is actually absolutely wild.
So, if you and I agree that the unknown (e.g. the future) is exciting and worth hanging around for, wouldn't that be kind of exciting too?
This is more or less also my take on religion. Nothing has convinced me of superior beings, but if it were to turn out there actually are some? Ho-ly shit.
I believe I have free will because it appears to me that I have free will.
Because I can't make a solid logical argument either way, I believe the thing that brings the greatest amount of joy to me. I never forget that there's an asterisk on this belief, though, and am ready to abandon it should an argument come along that removes ambiguity.
But I also believe that "belief is the death of intelligence", and so use "believe" as shorthand for "have as a working hypothesis".
Anyway the point is that just try to enjoy the ride if you can.
It doesn't matter to me if something is ultimately meaningless (such as the fact that in the end, this universe will cease to exist).
What matters to me is to have meaning in my own life. To feel as if my existence changed something. Hopefully, that "something" I've impacted is other people, and hopefully that impact is a positive one. To have people be better off for having known me.
That, for me, is real meaning.
How old are you? Because maybe we can reach immortality, but probably only if you are positioned properly for that. Anyway, for me the main drive is also curiosity on what is going to happen in the future. It's like this crazy story I want to know how it ends. I don't want to die, because I want to know what happens. Dying would be like going to sleep on a cliffhanger. I'll just prefer to binge rather than sleep.
The problem, however, is you’re effectively defining consciousness in a way that human brains do not follow unless you make appeals to some supernatural process beyond the reach of science (ie fundamentally subjective, unquantifiable).
So again, it’s a bulls*** definition of sentience.
I don't know what you mean by it being a BS definition of sentience. It's a technical term invented by philosophers to mean a specific thing. If you don't like it, use a different word.
Note: you brought up the term qualia, not me. I have never actually liked that term. I prefer the terms "consciousness" and "conscious experience".
If science is unable to account for the existence of such experiences, then either they fall outside of science (so what? who said empirical science was the only means of knowing? the claim itself isn't scientific) or science must be revised. And what of the claim that anything that isn't quantifiable is bullshit? Is that quantifiable? No? So it's bullshit?
If we're going let the AI revolution fulfill its potential, it's a cognitive shift we're going to have to embrace. After all, AI is going to render a lot of meat-based equipment obsolete and unnecessary, and someone's going to have to figure out how to dispose of all that wasteful junk.
What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.
I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.
My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461
Again (and again, and again), this argument is the mysteriously recurrent "many people fail", implying that failing would be normal, instead of undesirable.
> "parrots" things, as if that's not a pretty remarkable achievement
It is also the mark of unintelligence. As such, it is a warning alarm.
> pretty reliably
From a calculator you do not expect "«pretty»" reliable, but just reliable. From a human you expect faulty computation because of the economization of resources, but you know it can reason.
> Expecting perfect accuracy and knowledge is also not a reasonable expectation of an AI
The expectation is that of a reliable process bringing to the output.
I have seen plenty of bright and intelligence students who most of the time are just parrots of what were told in class or from some other sources. It is actually highly unusual for a high school student or even a freshman or sophomore in college to have an absolutely original and correct idea. That doesn't mean I look at my students as unintelligent, even the below-average ones.
And here comes the crux of the issue. You automatically assign the ability to "reason" to a human even when we make mistakes aplenty. But when a machine does it, you said that is a proof of unintelligence. This is what the OP was criticizing in their post. Most deniers are using arbitrary requirements applied only on the AI and not humans.
After teaching quite a few college level kids, I dare to say that the AI models right now exhibit reasoning ability close to many human. Doesn't mean they are sentient or human-level yet, but they are definitely much much better than what I would ever expect out of dumb silicon and steel.
Whereas I apply them requirements also to humans? :)
I must correct you, to help understanding: I do assign in general (I hope, to all individuals but for an n-th standard deviation) «the ability to "reason" to a human»; that «we make mistakes aplenty» is overriding (or under-using) a faculty which was present but, in those cases, was not exercised.
You write «when a machine does it»: but can the machine do it? Can it criticize its ontological contents, its built concepts? Humans can, and pretty much must - deontically. The machine should.
> we have to put in extra effort
Exactly. We can, many don't, and the machine should - just like people in general should.
> the problem you are trying to say
While people do not always exercise critical thinking, they should. Verifying, trying your ideas and concepts, making them solid, should be a constant effort. Humans have those modules, which makes them apt to possibly be employed as "predicting machines" (which is a definition of intelligence), or "things you can ask things".
When you build a machine that produces outputs, they may be "for fun", so every generator goes that satisfies the "fun" based "performance indicator", but more normally you will want the output to be reliable. So, the "verification" features are crucial.
> Something can be both undesirable and normal
In engineering, I hope not :)
If so, then you are right on the machine's part. The current LLMs can't change its own weights thus in a sense, incapable of questioning its own existence. It can't perform a highly philosophical debate with its own "self" and reform its mind as a result of that action.
What I am still arguing is many human can't do that either. Such thing, in my experience, is actually not that common in the general populace. And while you can say it is there, just "unexercised", that would be functionally identical to it not existing and you can't absolutely prove that "it is still there" in all human.
>In engineering, I hope not
Perhaps in your particular type of engineering. In civil engineering, material degradation is undesirable and normal. In chemical engineering, side reactions are undesirable and normal. In biomedical engineering, well there are too many to list, from undesirable epigenetic changes to patients' unhealthy lifestyles. Things can definitely be undesirable but normal.
Just have it use Wolfram Alpha plugin. Not an issue.
That doesn't really answer the question. Why is the immediate expectation, when AI is in its infancy, that it must outperform humans in the ability to be original and always factually accurate? That's not a reasonable expectation when there are not only no examples that it's even possible, but we ourselves aren't original. There's a reason why both computers and human beings have supervisors. What about AI suggests an intelligence that can or should be free of supervision?
> It is also the mark of unintelligence. As such, it is a warning alarm.
That depends on a matter of degree. You and I parrot things all the time. We neither have the time or capability to not take information for granted and pass it along without fully understanding it. What we do is best effort, which is far more achievable than "perfect" computation.
For instance, I could ask you to paint us a painting of literally anything, and chances are your painting will be based on your experience of what a "painting" is and what you paint will probably be fairly unremarkable on a grand scale, no matter how much you try. I don't mean this as an insult, because it applies to virtually everyone and it is what it is. Of course there is the possibility that you will prove me wrong and paint something that no one has painted before using a style or method that is completely new, but I won't bet on it.
In such a case, you'd be "parroting" what artists have been doing since time immemorial. Does that actually devalue your art? Or the art of practically every other artist who mostly painted things based on previous ideas? I don't think so, but that's just me.
> From a calculator you do not expect "«pretty»" reliable, but just reliable. From a human you expect faulty computation because of the economization of resources, but you know it can reason.
Yeah... because calculators don't reason anything.
> The expectation is that of a reliable process bringing to the output.
What is your best example of how the current generation of LLMs aren't reliable enough for a wide variety of tasks?
Well, having started in 1955, we really must frame that "infancy" in a scale like dog-years, human-years, AI-years...
> that it must outperform humans in the ability to be ... accurate
No, the expectation of some of us was not about «outperform[ing]», but about not losing the focus of «accura[cy]».
> a matter of degree
It does not seem a «matter of degree» but its opposite, a "matter of quality": does the machine question its statements?
> You and I
Actually, the process is that of building on already vetted ideas, and continuously vetting. True, there is an economy of effort, but there is also a capitalization on past effort. // It is unfair to say that you "«parrot»", just because of early stopping, assuming you are trained to vet your output.
> new ... what artists have been doing ... previous ideas
You should revise your perspective there: we learn from the (bigger and smaller) "giants" in the past, and learning is very different from plain imitation. What is learnt is internally elaborated, understood, integrated. There is a whole world of difference from "I learnt from A" and "Z copied A".
> calculators don't reason
But calculators bring a reliable result, and that is what is expected from them. You expect the same from intelligent entities.
> What is your best example
Decision Support Systems. (Edit: I meant as a support Analyst; as an information retriever it could instead be a dramatic asset, if employed as a hound to retrieve sources.)
Ah, I see. So when an AI imitates the things it was trained off of it's simply parroting the material. But when we imitate things we've trained off of, we're synthesizing new information that we've learned from. /s
This ambiguity is exactly what OP sounds like they're talking about in regards to intelligence. Who's to say what parroting is and what synthesizing new information from old ideas is? Where is the line? Can we score how "parroty" an idea is? Can we quantify how much intelligence is imbued in a written statement? There is no firm science (that I know of) that quantifies intelligence, but it sounds like we'll be needing to determine hard metrics to find out if AI is truly intelligent or not. Otherwise, we'll have these circular arguments forever that basically dismiss an AI as being intelligent simply because it's an AI.
Also in humans imitation as copying and imitation as intelligent inspiration are discriminated.
You see similar things in areas like science. When was the last time you’ve seen a paradigm shift akin to relativity or quantum mechanics? Not for over a century!
Question is - does this tool provide massive value and productivity boost? Yes.
And the «philosophical arguments» there highlight the other side, about the limitations. Are they relevant? Surely, if they are not evident to the user.
It can clearly do both of these things very well but human intelligence combines a multitude of interacting components and for tasks that require more than just the one it replicates it falls flat on its face.
It doesnt appear to be missing components that will can be fixed with incremental improvements to the model either.
The whole notion that computers or AI are capable of intelligence is simply a fantastical category mistake. A metaphor has been forgotten to be a metaphor. No matter how sophisticated AI technology could become in its apparent behavior, it cannot make the transition to bona fide intelligence. Simulation does not become the real McCoy through resemblance. This is just magical thinking. We must also account for the means by which the observable is produced. When we do, we can distinguish between a genuine behavior and a merely simulated behavior. And here is one important clue to the puzzle: there is a difference between true speech and something that merely happens to sound like speech, even when the raw data of each is indistinguishable to the ear per se. Observation is not brute. It always involves interpretation. When physicists conduct experiments, they are doing so within the context of some stack of theories[0] and background assumptions, including extra- and pre-scientific assumptions. The results of those experiments are observed which is to say read, and so observation is always theory- and assumption-laden because reading what we see or measure means taking what is made available to the senses as such and placing it in context. This is just a specific instance of what we all do in daily life (resting bitch face, or genuine irritation?). Science is nothing special in this regard.
So when you say that "anyone who has used ChatGPT and doesn't see that it can pretty reliably munge together different ideas is blind or is lying", I cannot help but respond with a "uh wut?". How have you come to conclude that ChatGPT has "ideas"? I will tell you that there is not a single idea within ChatGPT! Consider an example of an idea and see how we do. Take "triangularity", for instance. I think we can agree that "triangularity" is an idea. What makes "triangularity" different from some physical triangle? Well, there is only one "triangularity", but many triangles, each of which instantiates "triangularity" (otherwise, you could not call them all triangles; they are triangles only insofar as they instantiate some common "triangularity"). Each triangle is different from every other, and yet they are all triangles. And none of them is to be identified with "triangularity" as such, for if there is only one "triangularity", and some triangle were identical with "triangularity", then it would follow that there could only be one triangle! But, you might ask, is there perhaps a physical "triangularity" that isn't a triangle, yet something each triangle has? Well, no, because to be physical is to be concrete and particular, and "triangularity" is, by definition, general, universal, and abstract. There's no concrete magical ingredient that you can hold in your hand called "triangularity" that you can mix into cement to produce triangles.
There are other reasons why the claim that ChatGPT or whatever is intelligent isn't even wrong (like the subjectivity and observer relativity of computation in physical artifacts and computation as abstract model merely simulated through artifacts), but this appeal to abstraction, the ability to abstract from particulars and hold in your intellect what cannot exist physically except as particular, concrete instances, is central to intelligence. ChatGPT most certainly does not engage in abstraction. It would be fair to say that strictly speaking, it doesn't even parrot.
[0] https://en.wikipedia.org/wiki/Duhem%E2%80%93Quine_thesis
Firstly, we still don't truly have many good explanations for intelligence and consciousness that aren't self referential. Drawing hard conclusions on this subject about generative AI and human intelligence, even in 2023, is being hasty, in my opinion.
Which brings me to a lot of the criticisms made about LLMs. The idea that an ideal AI could or even should be capable of constantly generating original output is just plain weird. Where do people get this idea? How often do you meet a human being who hasn't gotten most of their ideas from somewhere else? Even synthesized ideas are forged by other ideas. Ideas that seem original may appear just that, and we just don't see where the connection lies between it and its predecessors. It's silly to dismiss an AI because it "parrots" things, as if that's not a pretty remarkable achievement. It's also very reductive; anyone who has used ChatGPT and doesn't see that it can pretty reliably munge together different ideas is blind or is lying about their use of ChatGPT.
Expecting perfect accuracy and knowledge is also not a reasonable expectation of an AI. An intelligence localized in spacetime is necessarily going to lack understanding of many things. To look to something like ChatGPT as a source of authoritative answers, and to dismiss it upon finding its answers to be specious, is to expect the AI to be God. Well I'm sorry, but we are nowhere near speaking directly to God.
In regards to "true understanding", for all we know, language processing is the workhorse of what humans consider to be acts of intelligence, and individual humans aren't as intelligent as they believe themselves to be. Our ability to communicate with each other about a variety of topics in real time is not so much a matter about being hyperaware of every given fact that is relevant and weighing them in a Bayesian model; how well we can converse is primarily dictated by how rapidly we can construct language by parsing the input, making connections, and producing output. I know I'm not the only one who can speak on a subject only to realize, after having completed a sentence, that I hardly even thought about what I was saying.