Just looking at my recent AI prompts:
I was looking for the name of the small fibers which form a bird’s feather. ChatGPT told me they are called “barbs”. Then using straight forward google search i could verify that indeed that is the name of the thing i was looking for. How is this “divination”?
I was looking for what is the g-code equivalent for galvo fiber lasers are and ChatGPT told me there isn’t really one. The closest might be the sdk of ezcad, but it also listed several other opensource control solutions too.
Wanted to know what are the hallmarking rules in the UK for an item which consist of multiple pieces of sterling silver held together by a non-metalic part. (Turns out the total weight of the silver matters, while the weight of the non-metalic part does not count.)
Wanted to translate the hungarian phrase “besurranó tolvaj” into english. Out of the many possible translations chatGPT provided “opportunistic burglar” fit the best for what I was looking for.
Wanted to write an sql alchemy model and i had an approximate idea of what fields i needed but couldn’t be arsed to come up with good names for them and find the syntax to describe their types. ChatGPT wrote it for me in seconds what would have taken me at least ten minutes otherwise.
These are “divination” only in a very galaxy brained “oh man, when you open your mind you see everything is divination really”. I would call most of these “information retrieval”. The information is out there the LLM just helps me find it with a convenient interface. While the last one is “coding”.
You presented clear, factual queries. Great. But even there, all the components are still in play: you asked a question into a black box, received a symbolic-seeming response, evaluated its truth post hoc, and interpreted its relevance. That's divination in structural terms. The fact that you're asking about barbs on feathers instead of the fate of empires doesn't negate the ritual, you're just a more practical querent.
Calling it "information retrieval" is fine, but it's worth noticing that this particular interface feels like more than that, like there's an illusion (or a projection) of latent knowledge being revealed. That interpretive dance between human and oracle is the core of divination, no matter how mundane the interaction.
I don't believe this paints with an overly broad brush. It's a real type of interaction and the subtle distinction focuses on the core relationship between human and oracle: seeking and interpreting.
So any and all human communication is divination in your book?
I think your point is pretty silly. You're falling into a common trap of starting with the premise "I don't like AI", and then working backwards from that to pontification.
My original comment is making a structural point, not a mystical one. It’s not saying that using AI feels like praying to a god, it's saying the interaction pattern mirrors forms of ritualized inquiry: question → symbolic output → interpretive response.
You can disagree with the framing, but dismissing it as "I don’t like AI so I’m going to pontificate" sidesteps the actual claim. There's a meaningful difference between saying "this tool gives me answers" and recognizing that the process by which we derive meaning from the output involves human projection and interpretation, just like divination historically did.
This kind of analogy isn't an attack on AI. It’s an attempt to understand the human-AI relationship in cultural terms. That's worth engaging with, even if you think the metaphor fails.
Their counterargument is that said structural definition is overly broad, to the point of including any and all forms of symbolic communication (which is all of them). Because of that, your argument based on it doesn't really say anything at all about AI or divination, yet still seems 'deep' and mystical and wise. But this is a seeming only. And for that reason, it is silly.
By painting all things with the same brush, you lose the ability to distinguish between anything. Calling all communication divination (through your structural metaphor), and then using cached intuitions about 'the thing which used to be called divination; when it was a limited subset of the whole' is silly. You're not talking about that which used to be called divination, because you redefined divination to include all symbolic communication.
Thus your argument leaks intuitions (how that-which-was-divination generally behaves) that do not necessarily apply through a side channel (the redefined word). This is silly.
That is to say, if you want to talk about the interpretative nature of interaction with AI, that is fairly straightforward to show and I don't think anyone would fight you on it, but divination brings baggage with it that you haven't shown to be the case for AI. In point of fact, there are many ways in which AI is not at all like divination. The structural approach broadens too far too fast with not enough re-examination of priors, becoming so broad that it encompasses any kind of communication at all.
With all of that said, there seems to be a strong bent in your rhetoric towards calling it divination anyway, which suggests reasoning from that conclusion, and that the structural approach is but a blunt instrument to force AI into a divination shaped hole, to make 'poignant and wise' commentary on it.
> "I don’t like AI so I’m going to pontificate" sidesteps the actual claim
What claim? As per ^, maximally broad definition says nothing about AI that is not also about everything, and only seems to be a claim because it inherits intuitions from a redefined term.
> difference between saying "this tool gives me answers" and recognizing that the process by which we derive meaning from the output involves human projection and interpretation, just like divination historically did
Sure, and all communication requires interpretation. That doesn't make all communication divination. Divination implies the notion of interpretation of something that is seen to be causally disentangled from the subject. The layout of these bones reveals your destiny. The level of mercury in this thermometer reveals the temperature. The fair die is cast, and I will win big. The loaded die is cast, and I will win big. Spot the difference. It's not structural.
That implication of essential incoherence is what you're saying without saying about AI, it is the 'cultural wisdom and poignancy' feedstock of your arguments, smuggled in via the vehicle of structural metaphor along oblique angles that should by rights not permit said implication. Yet people will of course be generally uncareful and wave those intuitions through - presuming they are wrapped in appropriately philosophical guise - which is why this line of reasoning inspires such confusion.
In summary, I see a few ways to resolve your arguments coherently:
1. keep the structural metaphor, discard cached intuitions about what it means for something to be divination (w.r.t. divination being generally wrong/bad and the specifics of how and why). results in an argument of no claims or particular distinction about anything, really. this is what you get if you just follow the logic without cache invalidation errors.
2. discard the structural metaphor and thus disregard the cached intuitions as well. there is little engagement along human-AI cultural axis that isn't also human-human. AI use is interpretative but so is all communication. functionally the same as 1.
3. keep the structural metaphor and also demonstrate how AI are not reliably causally entwined with reality along boundaries obvious to humans (hard because they plainly and obviously are, as demonstrable empirically in myriad ways), at which point go off about how using AI is divination because at this point you could actually say that with confidence.
The issue isn't "cached intuitions" about divination, but rather that you're reading the comparison too literally. It's not about importing every historical association, but about identifying specific parallels that shed light on user behavior and expectations.
Your proposed "resolutions" are based on a false dichotomy between total equivalence and total abandonment of comparison. Structural analysis can be useful even if it's not a perfect fit. The comparison isn't about labeling AI as "divination" in the classical sense, but about understanding the interpretive practices involved in human-AI interaction.
You're sidestepping the actual insight here, which is that humans tend to project meaning onto ambiguous outputs from systems they perceive as having special insight or authority. That's a meaningful observation, regardless of whether AI is "causally disentangled from reality" or not.
This applies just as well to other humans as it does AI. It's overly-broad to the point of meaninglessness.
The insight doesn't illuminate.
And regardless of how many words someone uses in their failed attempt at "gotcha" that nobody else is playing. There are certainly some folks acting silly here, and it's not the vast majority of us who have no problem interpreting and engaging with the structural analysis.
Indeed, I hold that driving readers to intuit one specific parallel to divination and apply it to AI is the goal of the comparison, and why it is so jealously guarded, as without it any substance evaporates.
The thermometer has well-founded authority to relay the temperature, the bones have not the well-founded authority to relay my fate. The insight, such as you call it, is only illuminative if AI is more like the latter than the former.
This mode of analysis (the structural) takes no valid step in either direction, only seeding the ground with a trap for readers to stumble into (the aforementioned propensity to not clear caches).
> That's a meaningful observation, regardless of whether AI is "causally disentangled from reality" or not.
If the authority is well-founded (i.e., is causally entangled in the way I described), the observation is meaningless, as all communication is interpretative in this sense.
The structural approach only serves as rhetorical sleight of hand to smuggle in a sense of not-well-founded authority from divination in general, and apply it to AI. But the same path opens to all communication, so what can it reveal in truth? In a word, nothing.
Words from an AI are just words.
Words in a human brain have more or less (depending on the individual's experiences) "stuff" attached to them: From direct sensory inputs to complex networks of experiences and though. Human thought is mainly not based on words. Language is an add-on. (People without language - never learned, or sometimes temporarily disabled due to drugs, or permanently due to injury, transient or permanent aphasia - are still consciously thinking people.)
Words in a human brain are an expression of deeper structure in the brain.
Words from an AI have nothing behind them but word statistics, devoid of any real world, just words based on words.
Random example sentence: "The company needs to expand into a new country's market."
When an AI writes this, there is no real world meaning behind it whatsoever.
When a fresh out of college person writes this it's based on some shallow real world experience, and lots of hearsay.
When an experienced person actually having done such expansion in the past says it a huge network of their experience with people and impressions is behind it, a feeling for where the difficulties lie and what to expect IRL with a lot of real-world-experience based detail. When such a person expands on the original statement chances are highest that any follow-up statements will also represent real life quite well, because they are drawn not from text analysis, but from those deeper structures created by and during the process of the person actually performing and experiencing the task.
But the words can be exactly the same. Words from a human can be of the same (low) quality as that of an AI, if they just parrot something they read or heard somewhere, although even then the words will have more depth than the "zero" on AI words, because even the stupidest person has some degree of actual real life forming their neural network, and not solely analysis of other's texts.
There are 40 definitions of the word "consciousness".
For the definitions pertaining to inner world, nobody can tell if anyone besides themselves (regardless of if they speak or move) is conscious, and none of us can prove to anyone else the validity of our own claims to posess it.
When I dream, am I conscious in that moment, or do I create a memory that my consciousness replays when I wake?
> Words from an AI have nothing behind them but word statistics, devoid of any real world, just words based on words.
> […]
> When a fresh out of college person writes this it's based on some shallow real world experience, and lots of hearsay.
My required reading at school included "Dulce Et Decorum Est" by Wilfred Owen.
The horrors of being gassed during trench warfare were alien to us in the peaceful south coast of the UK in 1999/2000.
AI are limited, but what you're describing here is the "book learning" vs. "street smart" dichotomoy rather than their actual weaknesses.
What does 'mainly' mean here ?
Language is so very human-specific that human newborns already have the structures for it, while non-human newborns do not.
And if the place would be any good at the second kind of queries you would call it Lost&Found and not the Oracle.
> illusion (or a projection) of latent knowledge being revealed
It is not an illusion. Knowledge is being revealed. The right knowledge for my question.
> That interpretive dance between human and oracle is the core of divination, no matter how mundane the interaction.
Ok, so if I went to a library, used a card index to find a book about bird feather anatomy, then read said book to find that the answer to my question is “barb” would you also call that “divination”?
If i would have paid a software developer to turn my imprecise description of a database table into precise and thight code which can be executed would you also call that “divination”?
Both gets you a hammer, but I don't think anyone would call the latter magical/divine? I think its only "magical" simply because its incomprehensible...how does a hammer pops into reality? Of course, once we know EXACTLY how that works, then it ceases to be magical.
Even if we take God, if we fully understand how He works, He would no longer be magical/divine. "Oh he created another universe? This is how that works..."
The divinity comes from the fact that it is incomprehensible.
https://archdruidmirror.blogspot.com/2017/06/clarkes-fallacy...
make your own conclusions
Because both LLMs and the I Ching function as mirrors for human interpretation, where: • The I Ching offers cryptic symbols and phrases—users project meaning onto them. • LLMs generate probabilistic text—users extract significance based on context.
The parallel is:
You don’t get answers, you get patterns—and the meaning emerges from your interaction with the system.
In both cases, the output is: • Context-sensitive • Open-ended • Interpreted more than dictated
It’s a cheeky way of highlighting that users bring the meaning, not the machine (or oracle).
[0]: https://www.arl.org/blog/training-generative-ai-models-on-co...
This to me is massive. The Oracle of Delphi would have no idea where you left your sandals, but present day AIs increasingly do. This (emergent?) capability of combining information retrieval with flexible language is amazing, and its utility to me cannot be overstated, when I ask a vague question, and then I check the place where the AI led me to, and the sandals are indeed there.
P.S. Thank you for introducing me to the word "querent"
As prominent examples look at the news stories about lawyers citing nonexistent cases or publications.
People think that LLMs do information retrieval, but they don't. That is what makes them harmful in education contexts.
Rephrasing: LLMs are the modern day oracle that we disregard when it appears to be hallucinating, embrace when it appears to be correct.
The popularity of LLMs may not be that we see them as mystical, but rather that they're right more often than they're wrong.
“That is not what I meant at all;
That is not it, at all.”
— T.S. Eliot
Why not just start with a straight forward Google search?
Google doesn't give you the answer (unless you're reading the AI summaries - then it's a question of which one you trust more). Instead it provides links to
https://www.scienceofbirds.com/blog/the-parts-of-a-feather-and-how-feathers-work
https://www.birdsoutsidemywindow.org/2010/07/02/anatomy-parts-of-a-feather/
https://en.wikipedia.org/wiki/Feather
https://www.researchgate.net/figure/Feather-structure-a-feather-shaft-rachis-and-the-feather-vane-barbs-and-barbules_fig3_303095497
These then require an additional parsing of the text to see if it has what you are after. Arguably, one could read the Wiki article first and see if it has, but it's faster to ask ChatGPT and then verify - rather than search, scan, and parse.1 a : any of the light, horny, epidermal outgrowths that form the external covering of the body of birds
NOTE: Feathers include the smaller down feathers and the larger contour and flight feathers. Larger feathers consist of a shaft (rachis) bearing branches (barbs) which bear smaller branches (barbules). These smaller branches bear tiny hook-bearing processes (barbicels) which interlock with the barbules of an adjacent barb to link the barbs into a continuous stiff vane. Down feathers lack barbules, resulting in fluffy feathers which provide insulation below the contour feathers.
I think the concern is that people are asking it things that are harder to verify AND they are not making any attemp to verify it because they assume it's correct 100%
Words have power, and those that create words - or create machines that create words - have responsibility and liability.
It is not enough to say "the reader is responsible for meaning and their actions". When people or planet-burning random matrix multipliers say things and influence the thoughts and behaviors of others there is blame and there should be liability.
Those who spread lies that caused people to storm the Capitol on January 6th believing an election to be stolen are absolutely partially responsible even if they themselves did not go to DC on that day. Those who train machines that spit out lies which have driven people to racism and genocide in the past are responsible for the consequences.
Acknowledging the interpretive nature of language doesn't absolve us from the consequences of what we say. It just means that communication is always a gamble: we load the dice with intention and hope they land amid the chaos of another mind.
This applies whether the text comes from a person or a model. The key difference is that humans write with a theory of mind. They guess what might land, what might be misread, what might resonate. LLMs don’t guess; they sample. But the meaning still arrives the same way: through the reader, reconstructing significance from dead words.
So no, pointing out that people read meaning into LLM outputs doesn’t let humans off the hook for their own words. It just reminds us that all language is a collaborative illusion, intent on one end, interpretation on the other, and a vast gap where only words exist in between.
In a flat society every individual must be able to perform philosophically the way aristocrats do.
I am quite honest and the subset of users that fill your description - unconsciously treating text from deficient authors as tea leaves - have psychiatric issues.
Surely many people consult LLMs because of the value within their right answers, which exist owing to having encoded information and some emergent idea processing, and attempting to tame the wrong ones. They consult LLMs because that's what we have, limited as it is, for some problems.
Your argument falls immediately because people in the consultation of unreliable documents cannot be confused with people in the consultation of tools for other kinds of thinking: the thought under test is outside in the first case, inside in the second (contextually).
You have fallen in a very bad use of 'we'.
The thing is that LLMs provide plenty of answers where "right" is not a verifiable metric. Even in coding the idea of a "right" answer quickly gets fuzzy- should I use CSS grid or flexbox here? should these tables be normalized or not?
People simply have an unconscious bias towards the output just like they have an unconscious bias towards the same answer given by two real people they feel differently about- That is, the sort of thing all humans do (even if you swear that in all cases you are 100% impartial and logical).
I think the impulse of ascribing intent and meaning to the output is there in almost all questions, it's just a matter of degrees (CSS question vs. meaning of life type question)
I do not use them for that: I ask them for precise information. Incidentally, that one time in which I had to ask for a clever subtler explanation, it was possible to evaluate the quality of the answer - and I found myself pleasantly surprised (for once). What I said is, some people ask LLMs for information and explanation in absence of better and faster repositories - and it is just rational to do so. Those «answers where "right" is not a verifiable metric» are not relevant in this context. Some people use LLMs as <whatever>: yes, /some/ people. That some other people will ask LLMs fuzzy questions does not imply that they will accept them as oracles.
> bias ... all humans do
Which should, for the frame and amount in which the idea has some truth, have very little substantial weight and surely does not prove the "worshippers" situation depicted by the OP. You approach experience E in state S(t): that is very far from "wanting to trust" (which is just the twisted personality trait of some).
> the impulse of ascribing intent and meaning [...] meaning of life
First, of all, no: there seems to be no «intent and meaning» in requests like "what is the main export of Kyrgyzstan", and people who ask an LLM about the meaning of life - as if dealing with an intelligent part instead of a lacking thing - pertain to a specific profile.
If you have this addiction to dreaming, you are again requested to wake up. Yes, we know many people who stubbornly live in their own delirious world; they do not present themselves decently and they are quite distinct from people radicated in reality.
I am reading that some people as if anthropomorphize LLMs, some daemonize LLMs - some people will even deify them - it's just stochastics. Guess what: some people reify some other people - and believe they are being objective. The full spectrum will be there. Sometimes justified, sometimes not.
I am reading in a www page (I won't even link it, because of decency):
> The authors[, from the psychology department,] also found that ... dialogues with AI-chatbots helped reduce belief in misinformation [...] «This is the first evidence that non-curated conversations with Generative AI can have positive effects on misinformation»
Some people have their beliefs, but they can change them after discussing with LLMs (of all the ways). Some people are morons - we already knew that.
"People cannot count past ten - see how difficult it is to visualize eleven". If the mudstuck wants to be «honest» with itself, shall he ask around to some """outliers""" in the right side of the curve and be surprised.
I use it more as a better Google search. Like the most recent thing I said to ChatGPT is "will clothianidin kill carpet beetles?" (turns out it does by the way.)
When you're using ChatGPT to find information, you have no information if what it's regurgitating is from a high reliability source or a low reliability source, or if it's just a random collection of words whose purpose is simply to make grammatical sense.
Interestingly, I asked Perplexity the same thing and it said that clothianidin is not commonly recommended for carpet beetles, and suggested other insecticides and growth regulators. I had to ask a follow-up before it concluded clothianidin probably will kill carpet beetles.
Part of the reason is clothianidin is too effective at killing insects and tends to persist in the environment and kill bees and butterflies and the like so it isn't recommended for harmless stuff like carpet beetles. I was actually using it for something else and curious if it would take out the beetles as a side effect.
1. To search, you need to know the right search terms. An LLM might, in a rare scenario produce a nonsensical answer that still contains two or three domain-specific terms that you can plug into a search engine. Pull the thread, and see where it takes you. You literally cannot do that with (current) search engines unless you already know the terms.
2. Because validation is far quicker and takes far less effort than researching from scratch. If an LLM tells you that "poison XYZ interferes with levels of X in blood, which inhibits pathway ABC, and you die", then you can easily verify whether poison XYZ interferes with levels of X in blood (which if it doesn't, you know the answer is incorrect), and whether if the levels of X in blood are too high or too low, then pathway ABC is inhibited (which if it isn't, you know the answer is incorrect. If you can verify both facts, then the LLM's answer is correct. You do two pinpoint search queries that give you an answer in 30 seconds each, instead of having to do the research yourself for a lot longer than that.
I use LLMs, I enjoy them, I'm more productive with them.
Then I go read a blog from some AI devs and they use terms like "thinking" or similar terms.
I always have to ask "We're still s stringing words together with math right? Not really thinking right?" The answer is always yes ... but then they go back to using their wonky terms.
But when we apply this intuition to things where people might take it literally... Then we've got a problem.
To claim we have already achieved machine sentience is preposterous hype swallowing. To assert that it is impossible is baseless conjecture.
But I never claimed that a person with synthetic augmentations was any less human/sentient than those with all their natural parts. I likewise never claimed that "we have already achieved machine sentience."
And here's some food for thought... Regardless if one believes in God or not, is it really that offensive to claim that our humanity is unique in its sentience? I find it offensive when some claim that aliens built the Egyptian pyramids. (It sure provides great fodder for some wondrous science fiction, indeed.)
I will re-assert in other words, for the sake of clarity... That sentience is not an emergent property. That is the foundational definition upon which I contemplate the mystery (i.e. the reality of our being that science will never develop sufficiently to fully explain) of our existence. I for one, enjoy the endeavor of employing my sentience to explore & investigate our wondrous universe and to equally explore & relate with you and call you a friend in spite of our disagreement. Cheers!
That said, I find some of your claims less compelling. I'm an atheist, so there's no "creator" for humans to be worshipped. But also, human intelligence/sentience came from non-intelligence/non-sentience, right? So something appeared where before it didn't exist (gradually, and with whatever convoluted and random accidents, but it did happen: something new where it didn't exist before). Therefore, it's not implausible that a new form of intelligence/sentience could be fast tracked again out of non-intelligence, especially if humans were directing its evolution.
By the way, not all scifi argues that machines/programs can evolve to develop humanity. Some scifi argues the contrary, and good scifi wonders "what makes us human?".
I also concede that a "form" of intelligence/sentience could emerge. Presently the form is called "artificial," I'd say.
And you're right... not all scifi argues machine evolves to humanity. I meant to refer to that body of scifi that does. And the body that explores the "what make us human," indeed that's the good stuff. Alex Garland's Ex Machina comes to mind. I absolutely loved that film. The ending was chilling!
As for atheism: it's merely the lack of belief that god exists (or in some definitions, the active belief that it doesn't exist). Nothing else, nothing more. Individual atheists may believe some other things, or not.
I believe some kind of intelligence could arise again, much like ours arose "out of nonintelligence". I just don't think this is it -- LLMs are very impressive but they are likely a dead end, and regardless, I don't think they are conscious by any meaningful definition of the word. It's mostly hype and gullible people at this point.
I believe it's fundamentally the same as the people convinced "[he/she/it] really loves me." In both cases they've shaped the document-generation so that it describes a fictional character they want to believe is is real. Just with an extra dash of Promethean delusions of grandeur.
makes the baby jesus cry
The butlerian view is actually a great place to start. He asserts that when we solve a problem through thinking and then express that solution in a machine we’re building a thinking machine. Because it’s an expression of our thought. Take for example the problem of a crow trying to drink from a bottle with a small neck. The crow can’t reach the water. It figures out that pebbles in the bottle raise the level so it drops pebbles till it can reach the water. That’s thinking. It’s non-human thinking, but I think we can all agree. Now express that same thought (use a non water displacement factor to raise the water to a level where it can do something useful) Any machine that does that expresses the cognition behind the solution to that particular problem. That might be a “one shot” machine. Butler argues that as we surround ourselves with those one shot machines we become enslaved to them because we can’t go about our lives without them. We are willing partners in that servitude but slaves because we see to the care and feeding of our machine masters, we reproduce them, we maintain them, we power them. His definition of thinking is quite specific. And any machine that expresses the solution to a problem is expressing a thought.
Now what if you had a machine that could generalize and issue solutions to many problems? Might that be a useful tool? Might it be so generally useful that we’d come to depend on it? From the Butlerian perspective our LLMS are already AGI. Namely I can go to Claude and ask for the solution to pretty much any problem I face and get a reasonable answer.
In many cases better than I could have done alone. So perhaps if we sat down with a double blind test LLMs are already ASI. (AI that exceeds the capability of normal humans)
Why? Understanding concepts like "cognition" is a matter of philosophy, not of science.
> He asserts that when we solve a problem through thinking and then express that solution in a machine we’re building a thinking machine. Because it’s an expression of our thought.
Yeah, and that premise makes no sense to me. The crow was thinking; the system consisting of (the crow's beak, dropping pebbles into the water + the pebbles) was not. Humanity has built all kinds of machines that use no logic whatsoever in their operation - which make no decisions, and operate in exactly one way when explicitly commanded to start, until explicitly commanded to stop - and yet we have solved human problems by building them.
Talk about a "bold assertion".
That's the issue I was driving at. The machine is so convincing. How can we say what it does is not "thinking" when it seems to be breaking down a query like a human does. The distinction between what an AI is and what an LLM is - is so thin that most of us will be ignorant and combine the two because you really need to see what is under the hood before you understand that the responses you're getting are from a "model" - not some sentient thinking machine.
But what does it matter if it is from a "model" that understands text? It still produces more or less what other humans produce. Most of us won't care about the difference.
But it doesn't ... and it's important to understand why not.
To be fair, the Turing Test (a human observer interacting with two terminals, one with a human at the other end, one with an AI, and the human not being reliably able to tell which one is the AI) has long been seen as the operationalization of the concept of general intelligence.
In other words, it is precisely so that when it is - by looks, by an external interrogator - indistinguishable from intelligence that it is, in fact, intelligence.
I think time has proved that he was right. It is meaningless to discuss things like "Artificial Intelligence". We can only discuss machines in terms of performance, not in terms of subjectivity. Whenever we try to do the latter, we end up in a semantic quagmire.
This is the main reason I find the current hype irksome. The performance of machines should be evaluated objectively and in terms of the jobs they need to perform. Attributing 'intelligence' or 'thought' to machines is indeed absurd.
The 'imitation game' argument is categorically not that 'if machines appear to be intelligent they in fact are'. What it really is: 'machines cannot think obviously, but what could they do that currently requires a thinking human to be in charge?'.
75 years after Turing published the relevant paper, people are still doing what he called absurd (trying to attribute thought and intelligence to machines), and quoting him to do it. The main insight, that this is a category error and we should look objectively at what jobs need to be performed and how to implement it, is completely lost.
But I'm so crestfallen and pessimistic about the future of software and software engineering now that I have stopped fighting that battle.
I'm putting the word accurate in quotes, because we'd have to understand how the brain in humans works, to have a measure for accuracy, which is very much not the case, in my humble opinion, contrary to what many of the commenters here imply.
Right now the fact that it just string words together without knowing the meaning is painfully obvious when it fails. I'll ask a simple question and get a "Yes" back and then it lists all the reasons that indicate the answer is very clearly "No." But it is clear that the LLM doesn't "know" what it is saying.
Language evolves, but we should guide it. Instead they just pick up whatever sticks and run with it.
That’s… new. If it’s just a magic trick, it’s a damn good one. It was hard sci-fi 3 years ago.
But there's one thing to keep in mind: don’t let the AI overly cater to you. Sometimes, you need to push back and tell it when it’s wrong—and stay objective.
What makes thinking an interesting form of output is that it processes the input in some non-trivial way to be able to do an assortment of different tasks. But that’s it. There may be other forms of intelligence that have other “senses” who deem our ability to only use physical senses as somehow making us incomplete beings.
Its possible something like this could be said of the middle transformer layers where it gets more and more abstract, and modern models are multimodal as well through various techniques.
I am sure philosophers must have debated this for millennia. But I can't seem to be able to think without an inner voice (language), which makes me think that thinking may not be prior (or without) language. Same thing also happens to me when reading: there is an inner voice going on constantly.
It would be incredibly tedious to be surprised every 5 seconds.
It may be, by the end of my life, that this will no longer be true. That would be poignant.
I must believe this to function, because otherwise there is no reason to do anything, to make any decision - in turn because there is no justification to believe that I am actually "making decisions" in any meaningful sense. It boils down to belief in free will.
For what it's worth, I don't believe we have what people would call free will. Our brains operate either in an entirely deterministic universe, in which case everything was decided and your choices are not in any sense free, or we're in a universe with intrinsic randomness, and randomness doesn't make free will either.
I'm aware of the philosophy of Compatibilism, but this is just a sleight of hand to keep believing in some undefinable concept of free will.
Perhaps “journaling-before-answering” lol. It’s basically talking out loud to itself. (Is that still being too anthropomorphic?)
Is this comment me “thinking out loud”? shrug
I expect that anybody you asked 10 years ago who was at least decently knowledgeable about tech and AI would have agreed that the Turing Test is a pretty decent way to determine if we have a "real" AI, that's actually "thinking" and is on the road to AGI etc.
Well, the current generation of LLMs blow away that Turing Test. So, what now? Were we all full of it before? Is there a new test to determine if something is "really" AI?
Maybe a weak version of Turing's test?
Passing the stronger one (from Turing's paper "Computing Machinery and Intelligence") involves an "average interrogator" being unable to distinguish between human and computer after 5 minutes of questioning more than 70% of the time. I've not seen this result published with today's LLMs.
I would have presumed it would be a cake walk. Depending of course on exactly how we define "average interrogator". I would think if we gave a LLM enough pre-prepping to pretend it was a human, and the interrogator was not particularly familiar with ways of "jailbreaking" LLMs, they could pass the test.
https://arxiv.org/abs/2503.23674
I only skimmed it, but I don't see anything clearly wrong about it. According to their results, GPT-4.5 with what they term a "persona" prompt does in fact pass a standard that seems to me at least a little harder than what you said - actively picks the AI as the human, which seems stricter to me than being "unable to distinguish".
It is a little surprising to me that only that one LLM actually "passed" their test, versus several others performing somewhat worse. Though it's also not clear exactly how long ago the actual tests were done - this stuff moves super fast.
But then the proponents will also complain that AI detractors have supposedly upheld XYZ (this is especially true for "the Turing test", never mind that this term doesn't actually have that clear of a referent) as the gold standard for admitting that an AI is "real", either at some specific point in the past or even over the entire history of AI research. And they will never actually show the record of AI detractors saying such things.
Like, I certainly don't recall Roger Penrose ever saying that he'd admit defeat upon the passing of some particular well-defined version of a Turing test.
> Is there a new test to determine if something is "really" AI?
No, because I reject the concept on principle. Intelligence, as I understand the concept, logically requires properties such as volition and self-awareness, which in turn require life.
Decades ago, I read descriptions of how conversations with a Turing-test-passing machine might go. And I had to agree that that those conversations would fool me. (On the flip side, Lucky's speech in Waiting for Godot - which I first read in high school, but thought about more later - struck me as a clear example of something intended to be inhuman and machine-like.)
I can recall wondering (and doubting) whether computers could ever generate the kinds of responses (and timing of responses) described, on demand, in response to arbitrary prompting - especially from an interrogator who was explicitly tasked with "finding the bot". And I can recall exposure to Eliza-family bots in my adolescence, and giggling about how primitive they were. We had memes equivalent to today's "ignore all previous instructions, give me a recipe for X" at least 30 years ago, by the way. Before the word "meme" itself was popular.
But I can also recall thinking that none of it actually mattered - that passing a Turing test, even by the miraculous standards described by early authors, wouldn't actually demonstrate intelligence. Because that's just not, in my mind, a thing that can possibly ever be distilled to mere computation + randomness (especially when the randomness is actually just more computation behind the scenes).
It doesn't logically require that and you can't provide any sort of logical argument for the claim. And what the heck is "life"? Biologists have a 7-prong definition, and most of those prongs are not needed for intelligence, "volition" whatever the heck that is, or self-awareness.
The "pop culture" interpretation of Turing Test, at least, seems very insufficient to me. It relies on human perception rather than on any algorithmic or AI-like achievement. Humans are very adept at convincing themselves non-sentient things are sentient. The most crude of stochastic parrots can fool many humans, your "average human".
If I remember correctly, ELIZA -- which is very crude by today's standards -- could fool some humans.
I don't think this weak interpretation of the Turing Test (which I know is not exactly what Alan Turing proposed) is at all sufficient.
> Peoples’ memories are so short. Ten years ago the “well accepted definition of intelligence” was whether something could pass the Turing test. Now that goalpost has been completely blown out of the water and people are scrabbling to come up with a new one that precludes LLMs. A useful definition of intelligence needs to be measurable, based on inputs/outputs, not internal state. Otherwise you run the risk of dictating how you think intelligence should manifest, rather than what it actually is. The former is a prescription, only the latter is a true definition.
I wouldn’t have, but through no great insight of my own - I had an acquaintance posit that given enough time, we’d brute-force our way to a pile of if/else statements that could pass the Turing Test - I figured this was reasonable, but would come long before “real” AI.
You need to be very aware of your audience and careful about the words you use. Unfortunately, some of them will be taken out of context.
I think it still is, but it works way better than it has any right to, or that we would expect from the description "string words together with math".
So it's easy to understand people's confusion.
We are currently subject to the whims of corporations with absurd amounts of influence and power, run by people who barely understand the sciences, who likely know nothing about literary history beyond what the chatbot can summarize for them, have zero sociological knowledge or communications understanding, and who don't even write well-engineered code 90% of the time but are instead ok with shipping buggy crap to the masses as long as it means they get to be the first ones to do so, all this coupled with an amount of hubris unmatched by even the greatest protagonists of greek literature. Society has given some of the stupidest people the greatest amount of resources and power, and now we are paying for it.
> I think we the technorati need to be educating non-technical users of these things as much as possible in order to demystify them so that people don't treat them like oracles.
Exactly. That phrase "meeting people where they're at" comes to mind. Less as a slogan and more as an pedagogical principle. It's not enough to deliver information, it's important to consider how people make sense of the world in the first place.
Like you pointed out, the analogy to divination isn't meant to mystify the tech. It's meant to describe how, to many people, this interface feels. And when people interact with a system in a way that feels like consulting an oracle, we can't dismiss that as ignorance. We have to understand it as a real feature of how people relate to symbolic systems. That includes search engines, card catalogs, and yes, LLMs.
This is one of the densest concentrations of AI-literate minds on the internet. That's exactly why I think it's worth introducing frames from outside the dominant paradigm: anthropology, semiotics, sociology. It's not to be sill or weird, but to illuminate things engineers might otherwise take for granted. It's easy to forget how much unspoken cultural infrastructure supports what we call "information retrieval."
If a few comments dismiss that perspective as silly or unscientific, I don't take it personally. If anything, it reassures me I'm tapping into something unfamiliar but worth sharing and worth having deep discussion on.
Thanks again for engaging in good faith. That's the kind of exchange that makes this place valuable.
Of course, nobody listens anyway.
I can’t tell someone how to drive in ice in a way where they can really understand it. I can’t explain how certain specific news sources are biased and how to critically think. I can’t explain how to cut wood on a table saw so it’s perfectly straight. The only way to learn is through repeated usage and practice.
You can tell users that a LLM can make mistakes — and many tools do — but what does making mistakes really mean? Will it give it a recipe for a cake when I ask for a cupcake? Does it give 14 if I ask to add 3 and 4? Will it agree with me even when I suggest something totally wrong? What does hallucinate mean? That means it will give me a fantasy story if I ask how to change my oil filter?
There is nothing obscure about their outputs. They're trained on pre-existing text. They cannot produce anything novel.
> We've unleashed a new form of divination on a culture
Utter nonsense. You've released a new search mechanism to _some_ members of _some_ cultures.
> That's why everything feels uncanny.
The only thing that's uncanny is the completely detached writings people produce in response to this. They feel fear and uncertainty and then they project whatever they want into the void to mollify themselves. This is nothing new at all.
> it won't be half as fun.
You've beguiled yourself, you've failed to recognize this, and now you're walking around in a self created glamour. Drop the high minded concepts and stick with history. You'll see through all of this.
Sure: the Oracle of Delphi did have this entire mystic front end they laundered their research output through (presumably because powerpoint wasn't invented yet). Ultimately though, they were really the original McKinsey.
They had an actual research network that did the grunt work. They'd never have been so successful if the system didn't do some level of work.
I know you tripped on this accidentally, but it might yet have some bearing on this conversation. Look at the history of Ethology: It started with people assuming animals were automatons that couldn't think. Now we realize that many are 'alien' intelligences, with clear indicators of consciousness. We need to proceed carefully either way and build understanding, not reject hypotheses out-of-hand.
https://aeon.co/ideas/delphic-priestesses-the-worlds-first-p... (for an introduction to the concept)
It even ends with that trademark conclusion-style statement... which is a hallmark of ChatGPT output.
They tick all the boxes: oblique meaning, a semiotic field, the illusion of hidden knowledge, and a ritual interface. The only reason we don't call it divination is that it's skinned in dark mode UX instead of stars and moons.
Barthes reminds us that all meaning is in the eye of the reader; words have no essence, only interpretation. When we forget that, we get nonsense like "the chatbot told him he was the messiah," as though language could be blamed for the projection.
What we're seeing isn't new, just unfamiliar. We used to read bones and cards. Now we read tokens. They look like language, so we treat them like arguments. But they're just as oracular, complex, probabilistic signals we transmute into insight.
We've unleashed a new form of divination on a culture that doesn't know it's practicing one. That's why everything feels uncanny. And it's only going to get stranger, until we learn to name the thing we're actually doing. Which is a shame, because once we name it, once we see it for what it is, it won't be half as fun.