They are doing useful stuff, saving time, etc, which can be measured. Thus also the defintion of AGI has largely become: "can produce or surpass the economic output of a human knowledge worker".
But I think this detracts from the more interesting discussion of what they are more essentially. So, while I agree that we should push on getting our terms defined, I think I'd rather work with a hazy definition, than derail so many AI discussion to mere economic output.
If you’re asking big questions like “can a machine think?” Or “is an AI conscious?” without doing the work of clarifying your concepts, then you’re only going to get vague ideas, sci-fi cultural tropes, and a host of other things.
I think the output question is also interesting enough on its own, because we can talk about the pragmatic effects of ChatGPT on writing without falling into this woo trap of thinking ChatGPT is making the human capacity for expression somehow extinct. But this requires one to cut through the hype and reactionary anti-hype, which is not an easy thing to do.
That is how I myself see AI: immensely useful new tools, but in no way some kind of new entity or consciousness, at least without doing the real philosophical work to figure out what that actually means.
IMO the issue is we won't be able to adequately answer this question before we first clearly describe what we mean of conscious thinking applied to ourselves. First we'd need to clearly define our own consciousness and what we mean by our own "conscious thinking" in a much, much clearer way than we currently do.
If we ever reach that point, I think we'd be able to fruitfully apply it to AI, etc., to assess.
Unfortunately we haven't been obstructed from answering this question about ourselves for centuries or millennia, but have failed to do so, so it's unlikely to happen suddenly now. Unless we use AIs to first solve that problem of defining our own consciousness, before applying it back on them. Which would be a deeply problematic order, since nobody would trust a breakthrough in the understanding of consciousness that came from AI, that is then potentially used to put them in the same class and define them as either thinking things or conscious things.
Kind of a shame we didn't get our own consciousness worked out before AI came along. Then again, wasn't for the lack of trying… Philosophy commanded the attention of great thinkers for a long time.
Do you think someone who has only ever studied pre-calc would be able to work through a calculus book if they had sufficient time? how about a multi-variable calc book? How about grad level mathematics?
IMO intelligence and thinking is strictly about this ratio; what can you extrapolate from the smallest amount of information possible, and why? From this perspective, I dont think any of our LLMs are remotely intelligent despite what our tech leaders say.
I would say a good definition has to, minimally, take on the Turing test (even if you disagree, you should say why). Or in current vibe parlance, it does "feel" intelligent to many people--they see intelligence in it. In my book this allows us to call it intelligent, at least loosely.
I have long thought this, but not had as good way to put it as you did.
If you think about geniuses like Einstein and ramanujen, they understood things before they had the mathematical language to express them. LLMs are the opposite; they fail to understand things after untold effort, training data, and training.
So the question is, how intelligent are LLMs when you reduce their training data and training? Since they rapidly devolve into nonsense, the answer must be that they have no internal intelligence
Ever had the experience of helping someone who's chronically doing the wrong thing, to eventually find they had an incorrect assumption, an incorrect reasoning generating deterministic wrong answers? LLMs dont do that; they just lack understanding. They'll hallucinate unrelated things because they dont know what they're talking about - you may have also had this experience with someone :)
This would be the equivalent of removing all senses of a human from birth and expecting them to somehow learn things. They will not. Therefore humans are not intelligent?
> LLMs dont do that; they just lack understanding.
You have no idea what they are doing. Since they are smaller than the dataset, they must have learned an internal algorithm. This algorithm is drawing patterns from somewhere - those are its internal, incorrect assumptions. It does not operate in the same way that a human does, but it seems ridiculous to say that it lacks intelligence because of that.
It sounds like you've reached a conclusion, that LLMs cannot be intelligent because they have said really weird things before, and are trying to justify it in reverse. Sure, it may not have grasped that particular thing. But are you suggesting that you've never met a human that is feigning understanding in a particular topic say some really weird things akin to an LLM? I'm an educator, and I have heard the strangest things that I just cannot comprehend no matter how much I dig. It really feels like shifting goalposts. We need to do better than that.
In split-brain experiments this is exactly how one half of the brain retroactively justifies the action of the other half. Maybe it is the case in LLMs that an overpowered latent feature sets the overall direction of the "thought" and then inference just has to make the best of it.
Simply put, to compare models, you describe both the model and training data using a code (usual reported as number of bits). The trained model that represents the data within the fewest number of bits is the more powerful model.
This paper [2] from ICML 2021 shows a practical approach for attempting to estimate MDL for NLP models applied to text datasets.
A crow bending a piece of wire into a hook to retrieve food demonstrates a novel solution extrapolated from minimal, non-instinctive, environmental input. This kind of zero-shot problem-solving aligns better with your definition of intelligence.
And yes, by this definition, LLMs pass with flying colours.
Firstly humans have not been evolving for “billions” of years.
Homo sapiens have been around for maybe 300’000 years, and the “homo” genus has been 2/3 million years. Before that we were chimps etc and that’s 6/7 million years ago.
If you want to look at the entire brain development, ie from mouse like creatures through to apes and then humans that’s 200M years.
If you want to think about generations it’s only 50/75M generations, ie “training loops”.
That’s really not very many.
Also the bigger point is this, for 99.9999% of that time we had no writing, or any kind of complex thinking required.
So our ability to reason about maths, writing, science etc is only in the last 2000-2500 years! Ie only roughly 200 or so generations.
Our brain was not “evolved” to do science, maths etc.
Most of evolution was us running around just killing stuff and eating and having sex. It’s only a tiny tiny amount of time that we’ve been working on maths, science, literature, philosophy.
So actually, these models have a massive, massive amount of training more than humans had to do roughly the same thing but using insane amounts of computing power and energy.
Our brains were evolved for a completely different world and environment and daily life that the life we lead now.
So yes, LLMs are good, but they have been exposed to more data and training time than any human could have unless we lived for 100000 years and still perform worse than we do in most problems!
Tell Boston Dynamics how to do that.
Mice inherited brain from their ancestors. You might think you don't need a working brain to reason about math, but that's because you don't know how thinking works, it's argument from ignorance.
People argue that humans have had the equivalent of training a frontier LLM for billions of years.
But training a frontier LLM involves taking multiple petabytes of data, effectively all of recorded human knowledge and experience, every book ever written, every scientific publication ever written, all of known maths, science, encylopedias, podcasts, etc. And then training that for millions of years worth of GPU-core time.
You cannot possibly equate human evolution with LLM training, it's ridiculous.
Our "training" time didn't involve any books, maths, science, reading, 99.9999% of our time was just in the physical world. So you can quite rationally argue that our brains ability to learn without training is radically better and more efficient that the training we do for LLMs.
Us running around in the jungle wasn't training our brain to write poetry or compose music.
This is a crux of your argument, you need to justify it. It sounds way off base to me. Kinda reads like an argument from incredulity.
Are you suggesting that somehow there were books in the jungle, or perhaps boardgames? Perhaps there was a computer lab in the jungle?
Were apes learning to conjugate verbs while munching on bananas?
I don't think I'm suggesting anything crazy here... I think people who say LLM training is equivalent to "billions of years of evolution" need to justify that argument far more than I need to justify that running around in the jungle is equivalent to mass processing petabytes of highly rich and complex dense and VARIED information.
One year of running around in the same patch of jungle, eating the same fruit, killing the same insects, and having sex with the same old group of monkeys isn't going to be equal to training with the super varied, complete, entirety of human knowledge, is it?
If you somehow think it is though, I'd love to hear your reasoning.
Language is one mode of expression, and humans have many. This is another factor that makes humans so effective. To be honest, I would say that physical observation is far more powerful than all the bodies of text, because it is comprehensive and can respond to interaction. But that is merely my opinion.
No-one should be arguing that an LLM training corpus is the same as evolution. But information comes in many forms.
I'm saying that LLM training is not equivalent to billions of years of evolution because LLMs aren't trained using evolutionary algorithms; there will always be fundamental differences. However, it seems reasonable to think that the effect of that "training" might be more or less around the same level.
I realise upon reading the OP's comment again that they may have been referring to "extrapolation", which is hugely problematic from the statistical viewpoint when you actually try to break things down.
My argument for compression asserts that LLMs see a lot of knowledge, but are actually quite small themselves. To output a vast amount of information in such a small space requires a large amount of pattern matching and underlying learned algorithms. I was arguing that humans are actually incredible compressors because we have many years of history in our composition. It's a moot point though, because it is the ratio of output to capacity that matters.
They can attempt to mimic the results for small instances of the problem, where there are a lot of worked examples in the dataset, but they will never ever be able to generalize and actually give the correct output for arbitrary sized instances of the problem. Not with current architectures. Some algorithms simply can't be expressed as a fixed-size matrix multiplication.
Were mammals the first thing? No. Earth was a ball of ice for a billion years - all life at that point existed solely around thermal vents at the bottom of the oceans... that's inside of you, too.
Evolution doesn't forget - everything that all life has ever been "taught" (violently had programmed into us over incredible timelines) all that has ever been learned in the chain of DNA from the single cell to human beings - its ALL still there.
When did I say that? Of course you look at a human's experience when you judge the quality of their output. And you also judge their output based on the context they did their work in. Newton wouldn't be Newton if he was the 14th guy to claim that the universe is governed by three laws of motion. Extending the example I used above, I would be more impressed if an art student aced a tough calc test than a math student, given that a math student probably has spent much more time with the material.
"Intelligence and "thinking" are abstract concepts, and I'm simply putting forward a way that I think about them. It works very much outside the context of AI too. The "smartest" colleagues I've worked with are somehow able to solve a problem with less information or time than I need. Its usually not because they have more "training data" than me.
1. A desire to learn calculus 2. A good teacher 3. No mental impairments such as dementia or other major brain drainers
could not learn calculus. Most people don't really care to try or don't get good resources. What you see as an intelligent mathematician is almost always someone born with better resources that was also encouraged to pursue math.
Nevertheless, we don’t have a good conceptual framework for thinking about these things, perhaps because we keep trying to apply human concepts to them.
The way I see it, a LLM crystallises a large (but incomplete and disembodied) slice of human culture, as represented by its training set. The fact that a LLM is able to generate human-sounding language
LLMs aren't Data (Star Trek) or Replicants (Blade Runner). They're not even David or the androids from the movie A.I.
It's general intelligence, not general humanity
The idea that we shouldn't use the word until further clarification is rather hilarious. Let's wait hundred years until somebody defines it?
It's not how words work. People might introduce more specific terms, of course. But the word already means what we think it means.
1. All previous discussion of thinking was in nature to human and animal minds. The reason this is a question in the first place right now is because we ostensibly have a new thing which looks like a human mind but isn’t. That’s the question at hand here.
2. The question in this particular topic is not about technological “progress” or anything like it. It’s about determining whether machines can think, or if they are doing something else.
3. There are absolutely instances in which the previous word doesn’t quite fit the new development. We don’t say that submarines are swimming like a fish or sailing like a boat. To suggest that “no, actually they are just swimming” is pretty inadequate if you’re trying to actually describe the new phenomenon. AIs and thinking seem like an analogous situation to me. They may be moving through the water just like fish or boats, but there is obviously a new phenomenon happening.
> if we could find characters or signs appropriate for expressing all our thoughts as definitely and as exactly as arithmetic expresses numbers or geometric analysis expresses lines, we could in all subjects in so far as they are amenable to reasoning accomplish what is done in arithmetic and geometry
2. You're missing the fact that meaning of words is defined through their use. It's an obvious fact that if people call certain phenomenon "thinking" then they call that "thinking".
3. The normal process is to introduce more specific terms and keep more general terms general. E.g. people doing psychometrics were not satisfied with "thinking", so they introduced e.g. "fluid intelligence" and "crystallized intelligence" as different kinds of abilities. They didn't have to redefine what "thinking" means.
The AI companies obviously want the masses to just assume these are intelligent beings who think like humans and so we can just trust their output as being truthful.
I have an intelligent IT colleague who doesn't follow the AI news at all and who has zero knowledge of LLMs, other than that our company recently allowed us limited Copilot usage (with guidelines as to what data we are allowed to share). I noticed a couple weeks ago that he was asking it various mathematical questions, and I warned him to be wary of the output. He asked why, so I asked him to ask copilot/chatGPT "how many r letters are in the word strawberry". Copilot initially said 2, then said after thinking about it, that actually it was definitely 3, then thought about it some more then said it can't say with reasonable certainty, but it would assume it must be 2. We repeated the experiment with completely different results, but the answer was still wrong. On the 3rd attempt, it got it right, though the "thinking" stages were most definitely bogus. Considering how often this question comes up in various online forums, I would have assumed LLM models would finally get this right but alas, here we are. I really hope the lesson instilled some level of skepticism to just trust the output of AI without first double-checking.
But that word can mean different things to different people. With no definition, how can you even begin to have a discussion around something?
Top-down theory of word definitions is just wrong. People are perfectly capable of using words without any formalities.
In any case, even if there isnt a _single_ definition of a given subject, in order to have a discussion around a given area, both sides need to agree on some shared understanding to even begin to debate in good faith in the first place. It's precisely this lack of definition which causes a breakdown in conversation in a myriad of different areas. A recent obvious (morbid) example would be "genocide".
Wittgenstein, who's considered one of most brilliant philosophers of XX century, in _Philosophical Investigations_ (widely regarded as the most important book of 20th-century philosophy) does not provide definitions, but instead goes through a series of examples, remarks, etc. In preface he notes that this structure is deliberate and he could not write it differently. The topic of the book includes philosophy of language ("the concepts of meaning, of understanding, of a proposition, of logic, the foundations of mathematics, states of consciousness,...").
His earlier book _Tractatus Logico-Philosophicus_ was very definition-heavy. And, obviously, Wittgenstein was well aware of things like dictionaries, and, well, all philosophical works up to that point. He's not the guy who's just slacking.
Another thing to note is that attempts to build AI using definitions of words failed, and not for a lack of trying. (E.g. Cyc project is running since 1980s: https://en.wikipedia.org/wiki/Cyc). OTOH LLMs which derive word meaning from usage rather than definition seems to work quite well.
I propose calling it "thunking"
Humans are special, we emit meaning the way stars emit photons, we are rare in the universe as far as empirical observation has revealed. Even with AGI the existence of each complex meaning generator will be a cosmic rarity.
For some people that seems to be not enough, due to their factually wrong word views they see themselves as common and worthless (when they empirically aren't) and need this little psychological boost of unexaminable metaphysical superiority.
But there is an issue of course, the type of thinking humans do is dangerous but net positive and relatively stable, we have a long history where most instantiations of humans can persist and grow themselves and the species as a whole, we have a track record.
These new models do not, people have brains that as they stop functioning they stop persisting the apparatus that supports the brain and they die, people tend to become less capable and active as their thinking deteriorates and hold less influence ocer others accept in rare cases.
This is not the case for an LLM, they seem to be able to hallucinate endlessly and as they have access to the outside world maintain roughly their same amount of causal leverage, their clarity and accuracy of their thinking isn't tied to their persisting.
Until we have empirical evidence to the contrary we need to preserve our species.
If we discover other smarter species or never do, either way I don't care, it's immaterial to the precautionary principle.
We are fucking awesome and rare, and any other species with our amount of meaning generation or even capability for meaning generation is also fucking awesome.
I would 100% grant that ceatacenas and octopuses have human or higher level intelligence, I don't care, I don't need to put other species capabilities down to highlight my species accomplishment, the simple fact is that we have written more facts about the universe, discovered more, done more, gone further than any species we have empirically observed.
I mean it's incontrovertibly true, maybe dolphins have crazy libraries I'm not aware of, but until we verify that fact we need to preserve ourselves (and afterwards too), and we should preserve them too.
Even with other species, aliens, etc, they all need to be preserved because we can't ex ante predict which entities within a species will be part of the causal chain that solves entropy (if it's even possible.)
That's prima facie absurd on the face of it, so I don't know what it means. You would have to a philosophical zombie to make such an argument.
you guys would have called lightsabers "laser swords" like Lucas originally did before Alec Guinness corrected him
How about Artificial Intelligence?
I still say it needs a new name. If we want to be generous, we could state "the limit as time approaches infinity of thunking, is thinking." (I don't believe we will ever achieve astronomically-superior AGI, and certainly don't believe it will ever have a will of its own that someone else didn't give it- which just makes it a tool.)
also, sorry but you (fellow) nerds are terrible at naming.
while "thunking" possibly name-collides with "thunks" from CS, the key is that it is memorable, 2 syllables, a bit whimsical and just different enough to both indicate its source meaning as well as some possible unstated difference. Plus it reminds me of "clunky" which is exactly what it is - "clunky thinking" aka "thunking".
And frankly, the idea it's naming is far bigger than what a "thunk" is in CS
If you do math in your head or math with a pencil/paper or math with a pocket calculator or with a spreadsheet or in a programming language, it is all the same thing.
The only difference with LLMs is the anthropomorphization of the tool.
Which doesn't really help because you can of course say that when you ask an LLM a question of opinion and it responds, it's having an opinion or that it's just predicting the next token and in fact has no opinions because in a lot of cases you could probably get it to produce the opposite opinion.
Same with the second definition - seems to really hinge on the definition of the word mind. Though I'll note the definitions for that are "The element of a person that enables them to be aware of the world and their experiences, to think, and to feel; the faculty of consciousness and thought." and "A person's intellect." Since those specify person, an LLM wouldn't qualify, though of course dictionaries are descriptive rather than prescriptive, so fully possible that meaning gets updated by the fact that people start speaking about LLMs as though they are thinking and have minds.
Ultimately I think it just... doesn't matter at all. What's interesting is what LLMs are capable of doing (crazy, miraculous things) rather than whether we apply a particular linguistic label to their activity.
It's possible there is much thinking that does not happen with written word. It's also possible we are only thinking the way LLMs do (by chaining together rationalizations from probable words), and we just aren't aware of it until the thought appears, whole cloth, in our "conscious" mind. We don't know. We'll probably never know, not in any real way.
But it sure seems likely to me that we trained a system on the output to circumvent the process/physics because we don't understand that process, just as we always do with ML systems. Never before have we looked at image classifications and decided that's how the eye works, or protein folding and decided that's how biochemistry works. But here we are with LLMs - surely this is how thinking works?
Regardless, I submit that we should always treat human thought/spirit as unknowable and divine and sacred, and that anything that mimics it is a tool, a machine, a deletable and malleable experiment. If we attempt to equivocate human minds and machines there are other problems that arise, and none of them good - either the elevation of computers as some kind of "super", or the degredation of humans as just meat matrix multipliers.
> It may be that this tech produces clear, rational, chain of logic writeups, but it's not clear that just because we also do that after thinking that it is only thinking that produces writeups.
I appreciate the way you describe this idea, I find it likely I'll start describing it the same way. But then you go on to write:
> Regardless, I submit that we should always treat human thought/spirit as unknowable and divine and sacred, and that anything that mimics it is a tool, a machine, a deletable and malleable experiment. If we attempt to equivocate human minds and machines there are other problems that arise, and none of them good - either the elevation of computers as some kind of "super", or the degredation of humans as just meat matrix multipliers.
Which I find to be the exact argument that you started by discarding.
It's not clear that equating organic, and synthetic thought will have any meaningful outcome at all, let alone worthy of baseless anxiety that it must be bad. Equally it seems absolutely insane to claim that anything is unknowable, and that because humanity doesn't have a clear foundational understanding that we should pretend that it's either devine, or sacred. Having spent any time watching the outcome of the thoughts of people, neither devine nor sacred are reasonable attributes to apply, but more importantly, I'd submit that you shouldn't be afraid to explore things you don't know, and you shouldn't advocate for others to adopt your anxieties.
I agree! I'm saying "If we equate them, we shortcut all the good stuff, e.g., understanding", because "it may be that this tech produces what we can, but that doesn't mean we are the same", which is good because it keeps us learning vs reducing all of "thinking" to just "Whatever latest chatgpt does". We have to continue to believe there is more to thinking, if only because it pushes us to make it better and to keep "us" as the benchmark.
Perhaps I chose the wrong words, but in essence what I'm saying is that giving up agency to a machine that was built to mimic our agency (by definition as a ML system) should be avoided at all costs.
Actually we have, several times. But the way we arrived at those conclusions is worth observing:
1. ML people figure out how the ML mechanism works.
2. Neuroscientists independently figure out how brains do it.
3. Observe any analogies that may or may not exist between the two underlying mechanisms.
I can't help but notice how that's a scientific way of doing it. By contrast, the way people tend to arrive at similar conclusions when talking about LLMs tends to consist of observing that two things are cosmetically similar, so they must be the same. That's not just pseudoscientific; it's the mode of reasoning that leads people to believe in sympathetic magic.
We spent decades slowly working towards this most recent sprint towards AI without ever landing on definitions of intelligence, consciousness, or sentience. More importantly, we never agreed on a way to recognize those concepts.
I also see those definitions as impossible to nail down though. At best we can approach it like disease - list a number of measurable traits or symptoms we notice, draw a circle around them, and give that circle a name. Then we can presume to know what may cause that specific list of traits or symptoms, but we really won't ever know as the systems are too complex and can never be isolated in a way that we can test parts without having to test the whole.
At the end of the day all we'll ever be able to say is "well it’s doing a thing that looks like thinking, ergo it’s thinking”. That isn't lazy, its acknowledging the limitations of trying to define or measure something that really is a fundamental unknown to us.
I have had this conversation too many times on HN. What I find astounding is the simultaneous confidence and ignorance on the part of many who claim LLMs are intelligent. That, and the occultism surrounding them. Those who have strong philosophical reasons for thinking otherwise are called "knee-jerk". Ad hominem dominates. Dunning-Kruger strikes again.
So LLMs produce output that looks like it could have been produced by a human being. Why would it therefore follow that it must be intelligent? Behaviorism is a non-starter, as it cannot distinguish between simulation and reality. Materialism [2] is a non-starter, because of crippling deficiencies exposed by such things as the problem of qualia...
Of course - and here is the essential point - you don't even need very strong philosophical chops to see that attributing intelligence to LLMs is simply a category mistake. We know what computers are, because they're defined by a formal model (or many equivalent formal models) of a syntactic nature. We know that human minds display intentionality[0] and a capacity for semantics. Indeed, it is what is most essential to intelligence.
Computation is a formalism defined specifically to omit semantic content from its operations, because it is a formalism of the "effective method", i.e., more or less procedures that can be carried out blindly and without understanding of the content it concerns. That's what formalization allows us to do, to eliminate the semantic and focus purely on the syntactic - what did people think "formalization" means? (The inspiration were the human computers that used to be employed by companies and scientists for carrying out vast but boring calculations. These were not people who understood, e.g., physics, but they were able to blindly follow instructions to produce the results needed by physicists, much like a computer.)
The attribution of intelligence to LLMs comes from an ignorance of such basic things, and often an irrational and superstitious credulity. The claim is made that LLMs are intelligent. When pressed to offer justification for the claim, we get some incoherent, hand-wavy nonsense about evolution or the Turing test or whatever. There is no comprehension visible in the answer. I don't understand the attachment here. Personally, I would find it very noteworthy if some technology were intelligent, but you don't believe that computers are intelligent because you find the notion entertaining.
LLMs do not reason. They do not infer. They do not analyze. They do not know, anymore than a book knows the contents on its pages. The cause of a response and the content of a response is not comprehension, but a production of uncomprehended tokens using uncomprehended rules from a model of highly-calibrated token correlations within the training corpus. It cannot be otherwise.[3]
[0] For the uninitiated, "intentionality" does not specifically mean "intent", but the capacity for "aboutness". It is essential to semantic content. Denying this will lead you immediately into similar paradoxes that skepticism [1] suffers from.
[1] For the uninitiated, "skepticism" here is not a synonym for critical thinking or verifying claims. It is a stance involving the denial of the possibility of knowledge, which is incoherent, as it presupposes that you know that knowledge is impossible.
[2] For the uninitiated, "materialism" is a metaphysical position that claims that of the dualism proposed by Descartes (which itself is a position riddled with serious problems), the res cogitans or "mental substance" does not exist; everything is reducible to res extensa or "extended substance" or "matter" according to a certain definition of matter. The problem of qualia merely points out that the phenomena that Descartes attributes exclusively to the former cannot by definition be accounted for in the latter. That is the whole point of the division! It's this broken view of matter that people sometimes read into scientific results.
[3] And if it wasn't clear, symbolic methods popular in the 80s aren't it either. Again, they're purely formal. You may know what the intended meaning behind and justification for a syntactic rule is - like modus ponens in a purely formal sense - but the computer does not.
Your premise is wrong.
Unless you want to claim that the distant cause by way of the training data is us, but that's exactly the conclusion you're trying to avoid. After all, we put the patterns in the training data, which means we already did the upfront intellectual work for the LLM.
> LLMs do not reason. They do not infer. They do not analyze.
(definitions from Oxford Languages)
reason(v): think, understand, and form judgments by a process of logic.
to avoid being circular, I'm willing to write this one off because of the 'think' and 'understand', as those are the root of the question here. However, forming a judgement by a process of logic is precisely what these LLMs do, and we can see that clearly in chain-of-logic LLM processes.
infer(v): deduce or conclude (information) from evidence and reasoning rather than from explicit statements.
Again, we run the risk of circular logic because of the use of 'reason'. An LLM is for sure using evidence to get to conclusions, however.
analyze(v): examine methodically and in detail the constitution or structure of (something, especially information), typically for purposes of explanation and interpretation.
This one I'm willing to go to bat for completely. I have seen LLM do this, precisely according to the definition above.
For those looking for the link to the above definitions - they're the snippets google provides when searching for "SOMETHING definition". They're a non-paywalled version of OED definitions.
Philosophically I would argue that it's impossible to know what these processes look like in the human mind, and so creating an equivalency (positive or negative) is an exercise in futility. We do not know what a human memory looks like, we do not know what a human thought looks like, we only know what the output of these things looks like. So the only real metric we have for an apples-to-apples comparison is the appearance of thought, not the substance of the thing itself.
That said, there are perceptible differences between the output of a human thought and what is produced by an LLM. These differences are shrinking, and there will come a point where we can no longer distinguish machine thinking and human thinking anymore (perhaps it won't be an LLM doing it, but some model of some kind will). I would argue that at that point the difference is academic at best.
Say we figure out how to have these models teach themselves and glean new information from their interactions. Say we also grant them directives to protect themselves and multiply. At what point do we say that the distinction between the image of man and man itself is moot?
I don't know how you arrived at that conclusion. This is no mystery. LLMs work by making statistical predictions, and even the word "prediction" is loaded here. This is not inference. We cannot clearly see it is doing inference, as inference is not observable. What we observe is the product of a process that has a resemblance to the products of human reasoning. Your claim is effectively behaviorist.
> An LLM is for sure using evidence to get to conclusions, however.
Again, the certainty. No, it isn't "for sure". It is neither using evidence nor reasoning, for the reasons I gave. These presuppose intentionality, which is excluded by Turing machines and equivalent models.
> [w.r.t. "analyze"] I have seen LLM do this, precisely according to the definition above.
Again, you have not seen an LLM do this. You have seen an LLM produce output that might resemble this. Analysis likewise presupposes intentionality, because it involves breaking down concepts, and concepts are the very locus of intentionality. Without concepts, you don't get analysis. I cannot understate the centrality of concepts to intelligence. They're more important than inference and indeed presupposed by inference.
> Philosophically I would argue that it's impossible to know what these processes look like in the human mind, and so creating an equivalency (positive or negative) is an exercise in futility.
That's not a philosophical claim. It's a neuroscientific one that insists that the answer must be phrased in neuroscientific terms. Philosophically, we don't even need to know the mechanisms or processes or causes of human intelligence to know that the heart of human intelligence is intentionality. It's implicit in the definition of what intelligence is! If you deny intentionality, you subject yourself to a dizzying array of incoherence, beginning with the self-refuting consequence that you could not be making this argument against intentionality in the first place without intentionality.
> At what point do we say that the distinction between the image of man and man itself is moot?
Whether something is moot depends on the aim. What is your aim? If you aim is theoretical, which is to say the truth for its own sake, and to know whether something is A or something is B and whether A is B, then it is never moot. If your aim is practical and scoped, if you want some instrument that has utility indistinguishable from or superior to that of a human being in the desired effects that it produces, then sure, maybe the question is moot in that case. I don't care if my computer was fabricated by a machine or a human being. I care about the quality of the computer. But then, in the latter case, you're not really asking whether there is a distinction between man and the image of man (which, btw, already makes the distinction that for some reason you want to forget or deny, as the image of a thing is never the same as the thing). So I don't really understand the question. The use of the word "moot" seems like a category mistake here. Besides, the ability to distinguish two things is an epistemic question, not an ontological one.
I ask the LLM to do some or another assessment. The LLM prints out the chain-of-thought (whether that moniker is accurate is academic - we can read the chain and see that at the very least, it follows a form recognizable as logic). At the end of the chain-of-thought, we are left with a final conclusion that the model has come to - a judgement. Whether the internal state of the machine looks anything like our own is irrelevant to these definitions, much like writing out a formalism (if A then B, if B then C, A implies C). Those symbols do not have any form save for the shape of them, but when used in accordance with the rules we have laid out regarding logic, they have meaning nonetheless.
I'd similarly push back against the idea that the LLM isn't using evidence - I routinely ask my LLMs to do so, and they search on the web, integrating the information gleaned into a cohesive writeup, and provide links so I can check their work. If this doesn't constitute "using evidence" then I don't know what does.
w.r.t. "analyze", I think you're adding some human-sauce to the definition. At least in common usage, we've used the term analyze to refer to algorithmic decoction of data for decades now - systems that we know for a fact have no intentionality other than directed by the user.
I think I can divine the place where our understandings diverge, and where we're actually on the same track. Per Dennet, I would agree with you that the current state of an LLM lacks intrinsic intention and thus certain related aspects of thought. Any intent must be granted by the user, at the moment.
However, it is on this point that I think we're truly diverging - whether it is possible for a machine to ever have intent. To the best of my understanding, animal intent traces it's roots to the biological imperative - and I think it's a bit of hubris to think that we can separate that from human intent. Now, I'm an empiricist before anything else, so I have to qualify this next part by saying it's a guess, but I suppose that all one needs to qualify for intent is a single spark - a directive that lives outside of the cognitive construct. For us, it lives in Maslow's hierarchy - any human intent can be traced back to some directive there. For a machine, perhaps all that's needed is to provide such a spark (along with a loop that would allow the machine to act without the prodding of the enter key).
I should apologize in advance, at this point, because I'm about to get even more pedantic. Still, I feel it relevant so let's soldier on...
As for whether the image of a thing is a thing, I ask this: is the definition of a thing, also that thing? When I use a phrase to define a chair, is the truth of the existence of that collection of atoms and energy contained within the word "chair", or my meaning in uttering it? Any idea that lives in words is constrained by the understanding of the speaker - so when we talk about things like consciousness and intentionality and reasoning we are all necessarily taking shortcuts with the actual Truth. It's for this reason that I'm not quite comfortable with laying out a solid boundary where empirical evidence cannot be built to back it up.
If I seem to be picking at the weeds, here, it's because I see this as an impending ethical issue. From what my meagre understanding can grok, there is a nonzero chance that we are going to be faced with determining the fate of a possibly conscious entity birthed from these machines in our lifetime. If we do not take the time to understand the thing and write it off as "just a machine", we risk doing great harm. I do not mean to say that I believe it is a foregone conclusion, but I think it right and correct that we be careful in examining our own presuppositions regarding the nature and scope of the thing. We have never had to question our understanding of consciousness in this way, so I worry that we are badly in need of practice.
This is a pretty messy argument as computers have been simulating material interactions for quite some time now.
Seems like you have that backwards. If consciousness is from a nonphysical process, like a soul that's only given to humans, then it follows that you can't build consciousness with physical machines. If it's purely physical, it could be built.
Your very idea is also predicated on the idea that it's possible for a real object to exist that isn't physical, and I think most modern philosophers reject the idea of a spiritual particle.
The simpler belief than the idea that just any intelligent process is conscious is that consciousness is some kind of physical process that happens in particular conditions.
I understand why computation doesn't intuitively seem the same as having a subjective conscious experience. But why is it any more intuitive that a bunch of protons, neutrons and electrons pushing and pulling on each other give rise to consciousness, but somehow do it by doing something other than computation?
If I impose a computational interpretation onto inert material after the fact, did that material possess the consciousness? Suppose I overlay a projection of the calculation of an AGI onto a circuit board, is that projection equally conscious to if that circuit board did the computation in the usual way?
No, that is absurd. The reasonable view is that computation is irrelevant and what is relevant is some special physical process.
But the alternative is equally absurd. In essence, you have a bunch protons, neutrons and electrons pushing and pulling on each other and somehow that causes consciousness. There just aren't that many weird properties or particles to appeal to as a cause of consciousness. While there are exotic particles and the quarks and gluons that make up protons/neutrons, these particles are relevant for our study of the universe, but they aren't relevant to understanding the human brain. Abstracting protons as a single particle that push and pull on other particles via forces works fine for explaining everything we know about the brain.
Not that it really changes anything if the particular way quarks interact did affect the brain in a way that couldn't be explained through the simplified view of a proton. It adds a few more particles to consider and the weirdness of quantum chromodynamics, but nothing there explains consciousness either.
So how do you go from particles pushing and pulling on each other to consciousness? It seems to me no matter how you arrange a bunch of particles, there is never any reason to assume that arrangement is conscious. It's just a bunch of points moving according to a few simple rules.
If you're saying there is something more in physics, some other force or particle or currently unknown way particles interact that causes consciousness, that's roughly the soul particle idea.
Saying consciousness is caused by a physical process is very vague. What kind of process? Why don't the physical processes the computers running an LLM perform count? Why does the brain qualify? If we go on to build many different types of artificial brains (using silicon or biological cells or anything else), how would you even go about recognizing which artificial thinking machines have the required physical process for consciousness?
>>>"Not that it really changes anything if the particular way quarks interact did affect the brain in a way that couldn't be explained through the simplified view of a proton. It adds a few more particles to consider and the weirdness of quantum chromodynamics, but nothing there explains consciousness either.
>>>So how do you go from particles pushing and pulling on each other to consciousness? It seems to me no matter how you arrange a bunch of particles, there is never any reason to assume that arrangement is conscious. It's just a bunch of points moving according to a few simple rules."
The pushing and pulling is another computational process. Consciousness must be some irreducible component of matter or of certain arrangements of matter, so that this matter "just is conscious". The pulling / pushing done in the correct way imposes a complex experience on that matter. I would love to tell you what it is, but my reasoning is a process of elimination, where we eliminate the popular idea that consciousness = computation now when it is very important to humanity to do so.
>>>"equally absurd".
I don't think it is, and I am very passionate about this. If computation is the only explainer of consciousness, then every large enough random collection of bits can be interpreted as representing some conscious process given a complicated enough interpretation rule set. E.g. take an exabyte of random data and extract from it a 1 mb ChatGPT conversation by deciding to include this bit and to exclude that bit. Who's to say that my rule for extracting the correct bits isn't a valid computational process? If I just write a random bit string over an exabyte memory bank over and over, and if some way of interpreting part of it at this frame, and some way of interpreting it in the next frame etc.. etc.. results in an intelligent conversation, was a conscious being simulated? Let's make it more absurd. Take a computation which expresses consciousness. Print out the conversation in bit form onto a piece of paper. Cut up and rearrange the paper into a sequence first of all the 0's and then all the 1's. Then show the result to your friend and say this can be interpreted at a conscious process, therefore is conscious. This is ridiculous. Why is a blank sheet of paper not a conscious process if some guy says by looking at the paper he imagines 50 0's in a row and 100 1's in a row, and those can be rearranged to express a thought in some manner?
>>>"how would you even go about recognizing which artificial thinking machines have the required physical process for consciousness?"
We would not be able to tell until we actually learn more physics, which is the deepest reason I think it is unethical to build them at all. My position on the physical nature of consciousness makes me believe that everything with a neuron-based brain probably has consciousness. And so if we are hell-bent on making something with consciousness, we could do it by growing a brain in a lab. That's not to say I believe only a brain can have consciousness, it's just that it is the only kind I will have confidence in for the time being. The reason I doubt our computer hardware has consciousness is that computation is abundant in the universe (basically anything can be interpreted as computation), and so I doubt that just any arbitrary hardware we've created is likely to interact the computation with the special sauce in the right way.
If that's all there is, it seems likely you could create consciousness out of marbles connected by springs. If protons and electrons don't have any special consciousness-related property other than pushing and pulling, we'd just have to recreate the arrangement of matter and the forces to get consciousness.
There is no way to make the leap from particles and forces to consciousness that seems intuitively right. No arrangement of dumb matter, whether marbles or protons, should be conscious.
On a somewhat related note: forces are rules that dictate how things move. At some level, position and movement is the only thing we can measure. When we measure something like voltage or charge, our measurement techniques actually rely on observing the movement of particles.
Consciousness doesn't seem like it's just movement. But if there were some other property of our universe that didn't cause movement, we would have no way to measure it. It would have no observable effect on matter. Which would make epiphenomenal consciousness seemingly impossible to prove or disprove even if it were some basic property of physics.
This exact argument applies to "free will", and that definition has been debated for millennia. I'm not saying don't try, but I am saying that it's probably a fuzzy concept for a good reason, and treating it as merely a behavioural descriptor for any black box that features intelligence and unpredictable complexity is practical and useful too.
Even in this thread, the number of people claiming some mystical power separating humans from all the rest of nature is quite noticeable.
Part of the issue is that our general concept of equality is limited by a first order classical logic which is a bad basis for logic
Thinking, like intelligence and many other words designating complex things, isn’t a simple topic. The word and concept developed in a world where it referred to human beings, and in a lesser sense, to animals.
To simply disregard that entire conceptual history and say, “well it’s doing a thing that looks like thinking, ergo it’s thinking” is the lazy move. What’s really needed is an analysis of what thinking actually means, as a word. Unfortunately everyone is loathe to argue about definitions, even when that is fundamentally what this is all about.
Until that conceptual clarification happens, you can expect endless messy debates with no real resolution.
“For every complex problem there is an answer that is clear, simple, and wrong.” - H. L. Mencken