News for hackers. Stuff that matters.
Re:What Happens When People Don't Understand How AI Works by tim333 on Monday June 09, 2025 @08:14AM (#44223690)
Maybe LLMs can be divinatory instruments but that sounds a bit highbrow going by my use.

I use it more as a better Google search. Like the most recent thing I said to ChatGPT is "will clothianidin kill carpet beetles?" (turns out it does by the way.)

Re:What Happens When People Don't Understand How AI Works by suddenlybananas on Monday June 09, 2025 @08:19AM (#44223722)
Until it makes stuff up.
Re:What Happens When People Don't Understand How AI Works by tim333 on Monday June 09, 2025 @08:26AM (#44223765)
Well nothing's perfect. It actually got that one wrong - it said "not typically effective" which isn't true.
Re:What Happens When People Don't Understand How AI Works by mort96 on Monday June 09, 2025 @09:35AM (#44224381)
Nothing is perfect, but some things let you validate the answer. Search engines give you search results, not an answer. You can use the traditional methods for evaluating the reliability of a resource. An academic resource focused on the toxicity of various kinds of chemicals is probably fairly trustworthy, while a blog from someone trying to sell you healing crystals probably isn't.

When you're using ChatGPT to find information, you have no information if what it's regurgitating is from a high reliability source or a low reliability source, or if it's just a random collection of words whose purpose is simply to make grammatical sense.

Re:What Happens When People Don't Understand How AI Works by AlecSchueler on Monday June 09, 2025 @09:52AM (#44224556)
The most frustrating time I've had is asking it something, pointing out why the answer was obviously wrong, having it confirm the faulty logic and give an example of a portion of the answer now it would look like if it had used sound logic, followed by a promise to answer again without the accepted inaccuracy, only to get an even more absurd answer than the first time around.
Re:What Happens When People Don't Understand How AI Works by Zak on Monday June 09, 2025 @08:30AM (#44223791)
This seems like the sort of question that's very likely to produce a hallucinated answer.

Interestingly, I asked Perplexity the same thing and it said that clothianidin is not commonly recommended for carpet beetles, and suggested other insecticides and growth regulators. I had to ask a follow-up before it concluded clothianidin probably will kill carpet beetles.

Re:What Happens When People Don't Understand How AI Works by tim333 on Monday June 09, 2025 @09:18AM (#44224209)
Yeah, as mentioned in another comment ChatGPT said "not generally effective" which I guess it hallucinated. It's actually a tricky question because the answer isn't really there in a straightforward way on the general web and I only know for sure because me and someone I know tried it. Although I guess a pesticide expert would find it easy.

Part of the reason is clothianidin is too effective at killing insects and tends to persist in the environment and kill bees and butterflies and the like so it isn't recommended for harmless stuff like carpet beetles. I was actually using it for something else and curious if it would take out the beetles as a side effect.

Re:What Happens When People Don't Understand How AI Works by thoroughburro on Monday June 09, 2025 @08:37AM (#44223850)
Trusting LLM advice about poisons seems… sort of like being a test pilot for a brand new aerospace company with no reputation for safety.
Re:What Happens When People Don't Understand How AI Works by tim333 on Monday June 09, 2025 @09:20AM (#44224225)
I agree in general but it wasn't of much importance whether my carpet beetles died or not.
Re:What Happens When People Don't Understand How AI Works by selfhoster11 on Monday June 09, 2025 @09:08AM (#44224125)
Only if you don't check it against a classical search query later. Not to mention that all you might get is search results from slop farms that aren't concerned with safety or factuality - ones that were a problem before LLMs.
Re:What Happens When People Don't Understand How AI Works by soraminazuki on Monday June 09, 2025 @08:07PM (#44231070)
So you agree that you need to do thorough research and be careful about which sources you trust. At that point, why not jump straight to actual research and bypass the information laundering machine entirely? Even you acknowledge that you can't trust what it says.
Re:What Happens When People Don't Understand How AI Works by selfhoster11 on Friday June 13, 2025 @09:58AM (#44268596)
Yes, I do. Why not jump into actual research from scratch then?

1. To search, you need to know the right search terms. An LLM might, in a rare scenario produce a nonsensical answer that still contains two or three domain-specific terms that you can plug into a search engine. Pull the thread, and see where it takes you. You literally cannot do that with (current) search engines unless you already know the terms.

2. Because validation is far quicker and takes far less effort than researching from scratch. If an LLM tells you that "poison XYZ interferes with levels of X in blood, which inhibits pathway ABC, and you die", then you can easily verify whether poison XYZ interferes with levels of X in blood (which if it doesn't, you know the answer is incorrect), and whether if the levels of X in blood are too high or too low, then pathway ABC is inhibited (which if it isn't, you know the answer is incorrect. If you can verify both facts, then the LLM's answer is correct. You do two pinpoint search queries that give you an answer in 30 seconds each, instead of having to do the research yourself for a lot longer than that.

Re:What Happens When People Don't Understand How AI Works by sorokod on Monday June 09, 2025 @08:58AM (#44224031)
Given the consistently declining quality of Google search, this is a low bar to pass.