News for hackers. Stuff that matters.
Re:What Happens When People Don't Understand How AI Works by krisoft on Sunday June 08, 2025 @07:20PM (#44220112)
This sounds very wise but doesn’t seem to describe any of my use cases. Maybe some use cases are divination but it is a stretch to call all of them that.

Just looking at my recent AI prompts:

I was looking for the name of the small fibers which form a bird’s feather. ChatGPT told me they are called “barbs”. Then using straight forward google search i could verify that indeed that is the name of the thing i was looking for. How is this “divination”?

I was looking for what is the g-code equivalent for galvo fiber lasers are and ChatGPT told me there isn’t really one. The closest might be the sdk of ezcad, but it also listed several other opensource control solutions too.

Wanted to know what are the hallmarking rules in the UK for an item which consist of multiple pieces of sterling silver held together by a non-metalic part. (Turns out the total weight of the silver matters, while the weight of the non-metalic part does not count.)

Wanted to translate the hungarian phrase “besurranó tolvaj” into english. Out of the many possible translations chatGPT provided “opportunistic burglar” fit the best for what I was looking for.

Wanted to write an sql alchemy model and i had an approximate idea of what fields i needed but couldn’t be arsed to come up with good names for them and find the syntax to describe their types. ChatGPT wrote it for me in seconds what would have taken me at least ten minutes otherwise.

These are “divination” only in a very galaxy brained “oh man, when you open your mind you see everything is divination really”. I would call most of these “information retrieval”. The information is out there the LLM just helps me find it with a convenient interface. While the last one is “coding”.

Re:What Happens When People Don't Understand How AI Works by kelseyfrog on Sunday June 08, 2025 @08:01PM (#44220274)
Sure, some people stepped up to the Oracle and asked how to conquer Persia. Others probably asked where they left their sandals. The quality of the question doesn't change the structure of the act.

You presented clear, factual queries. Great. But even there, all the components are still in play: you asked a question into a black box, received a symbolic-seeming response, evaluated its truth post hoc, and interpreted its relevance. That's divination in structural terms. The fact that you're asking about barbs on feathers instead of the fate of empires doesn't negate the ritual, you're just a more practical querent.

Calling it "information retrieval" is fine, but it's worth noticing that this particular interface feels like more than that, like there's an illusion (or a projection) of latent knowledge being revealed. That interpretive dance between human and oracle is the core of divination, no matter how mundane the interaction.

I don't believe this paints with an overly broad brush. It's a real type of interaction and the subtle distinction focuses on the core relationship between human and oracle: seeking and interpreting.

Re:What Happens When People Don't Understand How AI Works by gaganyaan on Sunday June 08, 2025 @10:22PM (#44220955)
> you asked a question into a black box, received a symbolic-seeming response, evaluated its truth post hoc, and interpreted its relevance

So any and all human communication is divination in your book?

I think your point is pretty silly. You're falling into a common trap of starting with the premise "I don't like AI", and then working backwards from that to pontification.

Re:What Happens When People Don't Understand How AI Works by kelseyfrog on Sunday June 08, 2025 @11:46PM (#44221273)
Hacker News deserves a stronger counterargument than “this is silly.”

My original comment is making a structural point, not a mystical one. It’s not saying that using AI feels like praying to a god, it's saying the interaction pattern mirrors forms of ritualized inquiry: question → symbolic output → interpretive response.

You can disagree with the framing, but dismissing it as "I don’t like AI so I’m going to pontificate" sidesteps the actual claim. There's a meaningful difference between saying "this tool gives me answers" and recognizing that the process by which we derive meaning from the output involves human projection and interpretation, just like divination historically did.

This kind of analogy isn't an attack on AI. It’s an attempt to understand the human-AI relationship in cultural terms. That's worth engaging with, even if you think the metaphor fails.

Re:What Happens When People Don't Understand How AI Works by hexaga on Monday June 09, 2025 @02:35AM (#44221928)
> Hacker News deserves a stronger counterargument than “this is silly.”

Their counterargument is that said structural definition is overly broad, to the point of including any and all forms of symbolic communication (which is all of them). Because of that, your argument based on it doesn't really say anything at all about AI or divination, yet still seems 'deep' and mystical and wise. But this is a seeming only. And for that reason, it is silly.

By painting all things with the same brush, you lose the ability to distinguish between anything. Calling all communication divination (through your structural metaphor), and then using cached intuitions about 'the thing which used to be called divination; when it was a limited subset of the whole' is silly. You're not talking about that which used to be called divination, because you redefined divination to include all symbolic communication.

Thus your argument leaks intuitions (how that-which-was-divination generally behaves) that do not necessarily apply through a side channel (the redefined word). This is silly.

That is to say, if you want to talk about the interpretative nature of interaction with AI, that is fairly straightforward to show and I don't think anyone would fight you on it, but divination brings baggage with it that you haven't shown to be the case for AI. In point of fact, there are many ways in which AI is not at all like divination. The structural approach broadens too far too fast with not enough re-examination of priors, becoming so broad that it encompasses any kind of communication at all.

With all of that said, there seems to be a strong bent in your rhetoric towards calling it divination anyway, which suggests reasoning from that conclusion, and that the structural approach is but a blunt instrument to force AI into a divination shaped hole, to make 'poignant and wise' commentary on it.

> "I don’t like AI so I’m going to pontificate" sidesteps the actual claim

What claim? As per ^, maximally broad definition says nothing about AI that is not also about everything, and only seems to be a claim because it inherits intuitions from a redefined term.

> difference between saying "this tool gives me answers" and recognizing that the process by which we derive meaning from the output involves human projection and interpretation, just like divination historically did

Sure, and all communication requires interpretation. That doesn't make all communication divination. Divination implies the notion of interpretation of something that is seen to be causally disentangled from the subject. The layout of these bones reveals your destiny. The level of mercury in this thermometer reveals the temperature. The fair die is cast, and I will win big. The loaded die is cast, and I will win big. Spot the difference. It's not structural.

That implication of essential incoherence is what you're saying without saying about AI, it is the 'cultural wisdom and poignancy' feedstock of your arguments, smuggled in via the vehicle of structural metaphor along oblique angles that should by rights not permit said implication. Yet people will of course be generally uncareful and wave those intuitions through - presuming they are wrapped in appropriately philosophical guise - which is why this line of reasoning inspires such confusion.

In summary, I see a few ways to resolve your arguments coherently:

1. keep the structural metaphor, discard cached intuitions about what it means for something to be divination (w.r.t. divination being generally wrong/bad and the specifics of how and why). results in an argument of no claims or particular distinction about anything, really. this is what you get if you just follow the logic without cache invalidation errors.

2. discard the structural metaphor and thus disregard the cached intuitions as well. there is little engagement along human-AI cultural axis that isn't also human-human. AI use is interpretative but so is all communication. functionally the same as 1.

3. keep the structural metaphor and also demonstrate how AI are not reliably causally entwined with reality along boundaries obvious to humans (hard because they plainly and obviously are, as demonstrable empirically in myriad ways), at which point go off about how using AI is divination because at this point you could actually say that with confidence.

Re:What Happens When People Don't Understand How AI Works by dingnuts on Monday June 09, 2025 @09:22AM (#44224238)
You're misunderstanding the point of structural analysis. Comparing AI to divination isn't about making everything equivalent, but about highlighting specific shared structures that reveal how humans interact with these systems. The fact that this comparison can be extended to other domains doesn't make it meaningless.

The issue isn't "cached intuitions" about divination, but rather that you're reading the comparison too literally. It's not about importing every historical association, but about identifying specific parallels that shed light on user behavior and expectations.

Your proposed "resolutions" are based on a false dichotomy between total equivalence and total abandonment of comparison. Structural analysis can be useful even if it's not a perfect fit. The comparison isn't about labeling AI as "divination" in the classical sense, but about understanding the interpretive practices involved in human-AI interaction.

You're sidestepping the actual insight here, which is that humans tend to project meaning onto ambiguous outputs from systems they perceive as having special insight or authority. That's a meaningful observation, regardless of whether AI is "causally disentangled from reality" or not.

Re:What Happens When People Don't Understand How AI Works by gaganyaan on Monday June 09, 2025 @09:42AM (#44224449)
> humans tend to project meaning onto ambiguous outputs from systems they perceive as having special insight or authority

This applies just as well to other humans as it does AI. It's overly-broad to the point of meaninglessness.

The insight doesn't illuminate.

Re:What Happens When People Don't Understand How AI Works by albedoa on Monday June 09, 2025 @09:44AM (#44224458)
> That's a meaningful observation, regardless of whether AI is "causally disentangled from reality" or not.

And regardless of how many words someone uses in their failed attempt at "gotcha" that nobody else is playing. There are certainly some folks acting silly here, and it's not the vast majority of us who have no problem interpreting and engaging with the structural analysis.

Re:What Happens When People Don't Understand How AI Works by hexaga on Monday June 09, 2025 @11:42PM (#44232244)
> It's not about importing every historical association, but about identifying specific parallels that shed light on user behavior and expectations.

Indeed, I hold that driving readers to intuit one specific parallel to divination and apply it to AI is the goal of the comparison, and why it is so jealously guarded, as without it any substance evaporates.

The thermometer has well-founded authority to relay the temperature, the bones have not the well-founded authority to relay my fate. The insight, such as you call it, is only illuminative if AI is more like the latter than the former.

This mode of analysis (the structural) takes no valid step in either direction, only seeding the ground with a trap for readers to stumble into (the aforementioned propensity to not clear caches).

> That's a meaningful observation, regardless of whether AI is "causally disentangled from reality" or not.

If the authority is well-founded (i.e., is causally entangled in the way I described), the observation is meaningless, as all communication is interpretative in this sense.

The structural approach only serves as rhetorical sleight of hand to smuggle in a sense of not-well-founded authority from divination in general, and apply it to AI. But the same path opens to all communication, so what can it reveal in truth? In a word, nothing.

Re:What Happens When People Don't Understand How AI Works by nosianu on Monday June 09, 2025 @06:19AM (#44222935)
> So any and all human communication is divination in your book?

Words from an AI are just words.

Words in a human brain have more or less (depending on the individual's experiences) "stuff" attached to them: From direct sensory inputs to complex networks of experiences and though. Human thought is mainly not based on words. Language is an add-on. (People without language - never learned, or sometimes temporarily disabled due to drugs, or permanently due to injury, transient or permanent aphasia - are still consciously thinking people.)

Words in a human brain are an expression of deeper structure in the brain.

Words from an AI have nothing behind them but word statistics, devoid of any real world, just words based on words.

Random example sentence: "The company needs to expand into a new country's market."

When an AI writes this, there is no real world meaning behind it whatsoever.

When a fresh out of college person writes this it's based on some shallow real world experience, and lots of hearsay.

When an experienced person actually having done such expansion in the past says it a huge network of their experience with people and impressions is behind it, a feeling for where the difficulties lie and what to expect IRL with a lot of real-world-experience based detail. When such a person expands on the original statement chances are highest that any follow-up statements will also represent real life quite well, because they are drawn not from text analysis, but from those deeper structures created by and during the process of the person actually performing and experiencing the task.

But the words can be exactly the same. Words from a human can be of the same (low) quality as that of an AI, if they just parrot something they read or heard somewhere, although even then the words will have more depth than the "zero" on AI words, because even the stupidest person has some degree of actual real life forming their neural network, and not solely analysis of other's texts.

Re:What Happens When People Don't Understand How AI Works by ben_w on Monday June 09, 2025 @07:21AM (#44223319)
> People without language - never learned, or sometimes temporarily disabled due to drugs, or permanently due to injury, transient or permanent aphasia - are still consciously thinking people.

There are 40 definitions of the word "consciousness".

For the definitions pertaining to inner world, nobody can tell if anyone besides themselves (regardless of if they speak or move) is conscious, and none of us can prove to anyone else the validity of our own claims to posess it.

When I dream, am I conscious in that moment, or do I create a memory that my consciousness replays when I wake?

> Words from an AI have nothing behind them but word statistics, devoid of any real world, just words based on words.

> […]

> When a fresh out of college person writes this it's based on some shallow real world experience, and lots of hearsay.

My required reading at school included "Dulce Et Decorum Est" by Wilfred Owen.

The horrors of being gassed during trench warfare were alien to us in the peaceful south coast of the UK in 1999/2000.

AI are limited, but what you're describing here is the "book learning" vs. "street smart" dichotomoy rather than their actual weaknesses.

Re:What Happens When People Don't Understand How AI Works by tempodox on Monday June 09, 2025 @08:48AM (#44223931)
I can only agree with you. And I find it disturbing that every time someone points out what you just said, the counter argument is to reduce human experience and human consciousness to the shallowest possible interpretation so they can then say, “look, it's the same as what the machine does”.
Re:What Happens When People Don't Understand How AI Works by kenjackson on Monday June 09, 2025 @10:37AM (#44224942)
I think it’s because the brain is simply a set of chemical and electrical interactions. I think some believe when we understand how the brain works it won’t be some “soulful” other worldly explanation. It will be some science based explanation that will seem very unsatisfying to some that think of us as more than complex machines. The human brain is different than LLMs, but I think we will eventually say “hey we can make a machine very similar”.
Re:What Happens When People Don't Understand How AI Works by tempodox on Monday June 09, 2025 @11:08AM (#44225276)
It looks like you did exactly what I described in my parent comment, so it doesn't add anything of substance. Let's agree to disagree.
Re:What Happens When People Don't Understand How AI Works by lcnPylGDnU4H9OF on Monday June 09, 2025 @05:14PM (#44229565)
The logic is that you preemptively shut down dissenting opinions so any comments with dissenting opinions are necessarily not adding anything of substance. They made good points and you simply don't want to discuss them; that does not mean the other commenter did not add substance and nuance to the discussion.
Re:What Happens When People Don't Understand How AI Works by tempodox on Tuesday June 10, 2025 @09:36AM (#44236652)
Nope. I understood the counterargument the first 513 times, there's no need to repeat it.
Re:What Happens When People Don't Understand How AI Works by kenjackson on Tuesday June 10, 2025 @12:04PM (#44238446)
Why bring up the argument then?
Re:What Happens When People Don't Understand How AI Works by rdtsc on Monday June 09, 2025 @02:48PM (#44227851)
The deconstruction trick is a bit like whataboutism. It sort of works on a shallow level but it's a cheap shot. You can say "this is just a collections of bites and matrix multiplications". If it's humans -- "it's just simple neurons firing and hormones". Even if it's some object: "what's the big deal, it's just bunch of molecules and atoms".
Re:What Happens When People Don't Understand How AI Works by BlueTemplar on Tuesday June 10, 2025 @10:32AM (#44237268)
> Human thought is mainly not based on words. Language is an add-on.

What does 'mainly' mean here ?

Language is so very human-specific that human newborns already have the structures for it, while non-human newborns do not.

Re:What Happens When People Don't Understand How AI Works by krisoft on Sunday June 08, 2025 @11:36PM (#44221229)
> some people stepped up to the Oracle and asked how to conquer Persia. Others probably asked where they left their sandals.

And if the place would be any good at the second kind of queries you would call it Lost&Found and not the Oracle.

> illusion (or a projection) of latent knowledge being revealed

It is not an illusion. Knowledge is being revealed. The right knowledge for my question.

> That interpretive dance between human and oracle is the core of divination, no matter how mundane the interaction.

Ok, so if I went to a library, used a card index to find a book about bird feather anatomy, then read said book to find that the answer to my question is “barb” would you also call that “divination”?

If i would have paid a software developer to turn my imprecise description of a database table into precise and thight code which can be executed would you also call that “divination”?

Re:What Happens When People Don't Understand How AI Works by anon-3988 on Monday June 09, 2025 @05:51AM (#44222803)
The difference is between saying '"I want a hammer" and it magically pops in your hand' versus '"I want a hammer" and I have to chop some wood, gather some metals, heat it up...'.

Both gets you a hammer, but I don't think anyone would call the latter magical/divine? I think its only "magical" simply because its incomprehensible...how does a hammer pops into reality? Of course, once we know EXACTLY how that works, then it ceases to be magical.

Even if we take God, if we fully understand how He works, He would no longer be magical/divine. "Oh he created another universe? This is how that works..."

The divinity comes from the fact that it is incomprehensible.

Re:What Happens When People Don't Understand How AI Works by BlueTemplar on Tuesday June 10, 2025 @10:11AM (#44237026)
Re:What Happens When People Don't Understand How AI Works by tough on Monday June 09, 2025 @06:34AM (#44222998)
I always like to compare tongue in cheek, llm's with I-ching

https://en.wikipedia.org/wiki/I_Ching

Re:What Happens When People Don't Understand How AI Works by AlecSchueler on Monday June 09, 2025 @09:47AM (#44224495)
Why?
Re:What Happens When People Don't Understand How AI Works by tough on Monday June 09, 2025 @09:55AM (#44224581)
i copy pasted my comment and your question to chatgpt, so this isnt my answer but the AI's:

make your own conclusions

Because both LLMs and the I Ching function as mirrors for human interpretation, where: • The I Ching offers cryptic symbols and phrases—users project meaning onto them. • LLMs generate probabilistic text—users extract significance based on context.

The parallel is:

You don’t get answers, you get patterns—and the meaning emerges from your interaction with the system.

In both cases, the output is: • Context-sensitive • Open-ended • Interpreted more than dictated

It’s a cheeky way of highlighting that users bring the meaning, not the machine (or oracle).

Re:What Happens When People Don't Understand How AI Works by narrator on Monday June 09, 2025 @07:13AM (#44223253)
You're describing what narrative archetype it is most similar to from ancient history, not what it actually is.
Re:What Happens When People Don't Understand How AI Works by zzbzq on Monday June 09, 2025 @08:18AM (#44223717)
The LLMs do have "latent knowledge," indisputably, the latent knowledge is beyond reproach. Because what we do know about the "black box" is that inside it, is a database of not just facts, but understanding, and we know the model "understands" nearly every topic better than any human. Where the doubt-worthy part happens is the generative step, since it is tasked with producing a new "understanding" that didn't already exist, the mathematical domain of the generative function exceeds the domain of reality. And, second of all, because the reasoning faculties are far less proven than the understanding faculties, and many queries require reasoning about existing understandings to derive a good, new one.
Re:What Happens When People Don't Understand How AI Works by datadrivenangel on Monday June 09, 2025 @08:21AM (#44223736)
LLMs have latent knowledge insofar as it can be distilled out of the internet...
Re:What Happens When People Don't Understand How AI Works by whilenot-dev on Monday June 09, 2025 @09:33AM (#44224362)
*or any digitized proprietary works, just as long as they can be parsed correctly. don't worry, the means of how to optain these works doesn't seem to matter[0]

[0]: https://www.arl.org/blog/training-generative-ai-models-on-co...

Re:What Happens When People Don't Understand How AI Works by falcor84 on Monday June 09, 2025 @10:54AM (#44225132)
> Others probably asked where they left their sandals.

This to me is massive. The Oracle of Delphi would have no idea where you left your sandals, but present day AIs increasingly do. This (emergent?) capability of combining information retrieval with flexible language is amazing, and its utility to me cannot be overstated, when I ask a vague question, and then I check the place where the AI led me to, and the sandals are indeed there.

P.S. Thank you for introducing me to the word "querent"

Re:What Happens When People Don't Understand How AI Works by sdenton4 on Monday June 09, 2025 @11:41AM (#44225607)
The particularly amazing part is that both the Oracle and the LLM said 'Right where you left them,' but only the LLM was correct.
Re:What Happens When People Don't Understand How AI Works by 725686 on Monday June 09, 2025 @11:15AM (#44225340)
"how to conquer Persia" and "what is the name of the small fibers which form a bird’s feather" are very different kinds of questions. There is no one right answer for the first. That is divination. The second is just information retrieval.
Re:What Happens When People Don't Understand How AI Works by diffeomorphism on Monday June 09, 2025 @12:16PM (#44226019)
Which the LLM then does not do and instead makes up likely text.

As prominent examples look at the news stories about lawyers citing nonexistent cases or publications.

People think that LLMs do information retrieval, but they don't. That is what makes them harmful in education contexts.

Re:What Happens When People Don't Understand How AI Works by navane on Monday June 09, 2025 @01:49AM (#44221717)
What about the times you didn't get a coherent answer and you gave up and looked elsewhere?
Re:What Happens When People Don't Understand How AI Works by JKCalhoun on Monday June 09, 2025 @08:15AM (#44223700)
Almost proves it is not an oracle then, not perceived as one.

Rephrasing: LLMs are the modern day oracle that we disregard when it appears to be hallucinating, embrace when it appears to be correct.

The popularity of LLMs may not be that we see them as mystical, but rather that they're right more often than they're wrong.

“That is not what I meant at all;

That is not it, at all.”

— T.S. Eliot

Re:What Happens When People Don't Understand How AI Works by dfxm12 on Monday June 09, 2025 @08:15AM (#44223695)
ChatGPT told me they are called “barbs”. Then using straight forward google search i could verify that indeed that is the name of the thing i was looking for.

Why not just start with a straight forward Google search?

Re:What Happens When People Don't Understand How AI Works by shagie on Monday June 09, 2025 @08:29AM (#44223784)
If you are not familiar with the term, it can be hard to search for it.

Google doesn't give you the answer (unless you're reading the AI summaries - then it's a question of which one you trust more). Instead it provides links to

    https://www.scienceofbirds.com/blog/the-parts-of-a-feather-and-how-feathers-work
    https://www.birdsoutsidemywindow.org/2010/07/02/anatomy-parts-of-a-feather/
    https://en.wikipedia.org/wiki/Feather
    https://www.researchgate.net/figure/Feather-structure-a-feather-shaft-rachis-and-the-feather-vane-barbs-and-barbules_fig3_303095497
    
These then require an additional parsing of the text to see if it has what you are after. Arguably, one could read the Wiki article first and see if it has, but it's faster to ask ChatGPT and then verify - rather than search, scan, and parse.
Re:What Happens When People Don't Understand How AI Works by ramchip on Monday June 09, 2025 @08:41AM (#44223875)
It gives you more effective search keywords. "Fibers in feathers" isn't too bad, but when it's quite vague like "that movie from the 70s where the guy drank whiskey and then there was a firefight and..." getting the name from the LLM makes it much faster to google.
Re:What Happens When People Don't Understand How AI Works by bandoti on Monday June 09, 2025 @09:15AM (#44224184)
Funny I just entered “feather” into Merriam-Webster dictionary and there’s your word “barb”. Point being, people should use a dictionary/thesaurus before burning fuel on an AI.

1 a : any of the light, horny, epidermal outgrowths that form the external covering of the body of birds

NOTE: Feathers include the smaller down feathers and the larger contour and flight feathers. Larger feathers consist of a shaft (rachis) bearing branches (barbs) which bear smaller branches (barbules). These smaller branches bear tiny hook-bearing processes (barbicels) which interlock with the barbules of an adjacent barb to link the barbs into a continuous stiff vane. Down feathers lack barbules, resulting in fluffy feathers which provide insulation below the contour feathers.

Re:What Happens When People Don't Understand How AI Works by dingnuts on Monday June 09, 2025 @09:18AM (#44224205)
This is a great example because the LLM answer was insufficiently complete but if you didn't look up the result you wouldn't know. I think I remain an AI skeptic because I keep looking up the results and this kind of omission is more common than not.
Re:What Happens When People Don't Understand How AI Works by chipsrafferty on Thursday June 12, 2025 @01:56PM (#44260633)
> Then using straight forward google search i could verify

I think the concern is that people are asking it things that are harder to verify AND they are not making any attemp to verify it because they assume it's correct 100%