News for hackers. Stuff that matters.
Re:Google Is Done by Tepix on Thursday December 01, 2022 @11:27AM (#33818490)
Exactly, i talked to ChatGPT and it gave me a lot of wrong information in an authorative tone. I consider it dangerous as-is.
Re:Google Is Done by adamsmith143 on Thursday December 01, 2022 @11:29AM (#33818533)
Turns out humans do this all the time and they actually have real power.
Re:Google Is Done by Tepix on Thursday December 01, 2022 @11:33AM (#33818617)
Yes, just look at the twitter thread, most people are not even noticing that the answers are wrong.
Re:Google Is Done by jerf on Thursday December 01, 2022 @12:00PM (#33819062)
Yes they do, and I do not deny the power of human's ability to confidently spew nonsense.

However, humans do have some known failure cases that help us detect that. For instance, pressing the human on a couple of details will generally show up all but the very best bullshit artists; there is a limit to how fast humans can make crap up. Some of us are decent at the con-game aspects but it isn't too hard to poke through this limit on how fast they can make stuff up.

Computers can confabulate at full speed for gigabytes at a time.

Personally, I consider any GPT or GPT-like technology unsuitable for any application in which truth is important. Full stop. The technology fundamentally, in its foundation, does not have any concept of truth, and there is no obvious way to add one, either after the fact or in its foundation. (Not saying there isn't one, period, but it certainly isn't the sort of thing you can just throw a couple of interns at and get a good start on.)

"The statistically-most likely conclusion of this sentence" isn't even a poor approximation of truth... it's just plain unrelated. That is not what truth is. At least not with any currently even remotely feasible definition of "statistically most likely" converted into math sufficient to be implementable.

And I don't even mean "truth" from a metaphysical point of view; I mean it in a more engineering sense. I wouldn't set one of these up to do my customer support either. AI Dungeon is about the epitome of the technology, in my opinion, and generalized entertainment from playing with a good text mangler. It really isn't good for much else.

Re:Google Is Done by visarga on Thursday December 01, 2022 @12:07PM (#33819165)
> Personally, I consider any GPT or GPT like technology unsuitable for any application in which truth is important . Full stop. The technology fundamentally, in its foundation, does not have any concept of truth

I think you got it all wrong. Not all GPT-3 tasks are "closed-book".

If you can fit in the context a piece of information, then GPT-3 will take it into consideration. That means you can do a search, get the documents into the prompt, and then ask your questions. It will reference the text and give you grounded answers. Of course you still need to vet the sources of information you use, if you give it false information into the context, it will give wrong answers.

Re:Google Is Done by jerf on Thursday December 01, 2022 @01:05PM (#33820044)
I don't think you're right. Even if you add "correct" context, and in many of these cases "I can locate correct context" already means the GPT-tech isn't adding much, GPT still as absolutely no guard rails stopping it from confabulating. It might confabulate something else, but it still confabulating.

Fundamentally, GPT is a technology for building convincing confabulations, and we hope that if we keep pounding on it and making it bigger we can get those confabulations to converge on reality. I do not mean this as an insult, I mean it as a reasonable description of the underlying technology. This is, fundamentally, not a sane way to build most of the systems I see people trying to build with it. AI Dungeon is a good use because the whole point of AI Dungeon is to confabulate at scale. This works with the strengths of GPT-like tech (technically, "transformer-based tech" is probably a closer term but nobody knows what that is).

Re:Google Is Done by mannykannot on Thursday December 01, 2022 @02:18PM (#33821231)
This hangs on what it means to "take it into consideration." If you gave me new information, I would attempt to see it in context, evaluate its relevance, and either update my positions accordingly or explain why I do not see it making a difference. If I saw difficulties doing this, I would ask for clarification, explaining what it was that was seemed difficult or unclear.

As far as I can tell, there is no reason to think that the way GPT-3 generates its responses will result in this happening - even the basic ability of correctly inferring corollaries from a collection of facts seems beyond what those methods could deliver, except insofar as the syntax of their expression matches common patterns in the corpus of human language use. And the empirical results so far, while being impressive and thought-provoking in many ways, support this skepticism.

Re:Google Is Done by adamsmith143 on Thursday December 01, 2022 @12:21PM (#33819373)
>Computers can confabulate at full speed for gigabytes at a time.

This I think is the actual problem. Online forums will likely be filled with AI generated BS in the very near future, if not already.

>"The statistically-most likely conclusion of this sentence" isn't even a poor approximation of truth... it's just plain unrelated. That is not what truth is. At least not with any currently even remotely feasible definition of "statistically most likely" converted into math sufficient to be implementable.

It's not necessarily clear that this isn't what Humans are doing when answering factual questions.

>And I don't even mean "truth" from a metaphysical point of view; I mean it in a more engineering sense. I wouldn't set one of these up to do my customer support either. AI Dungeon is about the epitome of the technology, in my opinion, and generalized entertainment from playing with a good text mangler. It really isn't good for much else.

By the same logic how can we allow Humans to do those jobs either? How many times has some distant call center person told you "No sir there is definitely no way to fix this problem" when there definitely was and the person was just ignorant or wrong? We should be more concerned with getting the error rate of these AI systems to human level or better, which they already are in several other domains so it's not clear they won't get to that level soon.

Re:Google Is Done by jerf on Thursday December 01, 2022 @01:03PM (#33820006)
"By the same logic how can we allow Humans to do those jobs either?"

First, since you can't see tone, let me acknowledge this is a fair question, and this answer is in the spirit of exploration and not "you should have known this" or anything like that.

The answer is a spin on what I said in my first post. Human failures have a shape to them. You cite an example that is certainly common, and you and I know what it means. Or at least, what it probabilistically means. It is unfortunate if someone with lesser understanding calls in and gets that answer, but at least they can learn.

If there were a perfect support system, that would be preferable, but for now, this is as good as it gets.

A computer system will spin a much wider variety of confabulated garbage, and it is much harder to tell the difference between GPT text that is correct, GPT text that is almost correct but contains subtle errors, and GPT text that sounds very convincing but is totally wrong. The problem isn't that humans are always right and computers are always wrong; the problem is that the bar for being able to tell if the answer is correct is quite significantly raised for me as someone calling in for GPT-based technologies.

Re:Google Is Done by lancesells on Thursday December 01, 2022 @01:30PM (#33820431)
The thing I think about GPT and tools like Stable Diffusion is do we as humanity need it? Do they add any value to our current world outside of an achievement in computer science? I don't think so but would love to hear arguments about needing it.
Re:Google Is Done by adamsmith143 on Thursday December 01, 2022 @01:37PM (#33820545)
Did we need digital painting tools? Paint and Easel worked just fine. Did we need paint and easels? Cave walls and clay pigments worked just fine. Do we need Automobiles or Trains? Horses worked just fine. Etc. Etc. Etc.
Re:Google Is Done by majormajor on Thursday December 01, 2022 @12:53PM (#33819848)
So if you're suggestion is that it's ok if computers are as unreliable as humans, what's the point of computers, then?
Re:Google Is Done by rafaelero on Thursday December 01, 2022 @12:55PM (#33819887)
The point is that they get better and they don't need to be perfect.
Re:Google Is Done by majormajor on Thursday December 01, 2022 @01:43PM (#33820637)
Nobody's shown a way yet to teach a computer how to tell bullshit from facts and filter out the bullshit in it's regurgitation/hallucination text creation stuff.

So until that happens, all you've done is let people put bullshit-spewing humans in more places. People already know not to necessarily trust humans, now they'll (re)learn that about computer generated text. (It's actually probably not clear to everyone what's computer-generated text and human-generated text, so more likely, specific places that rely on this will just be seen as untrustworthy. "Create more untrustworthy sources of text" is... underwhelming, honestly.)

Re:Google Is Done by rafaelero on Thursday December 01, 2022 @01:58PM (#33820884)
> Nobody's shown a way yet to teach a computer how to tell bullshit from facts and filter out the bullshit in it's regurgitation/hallucination text creation stuff.

And yet they keep improving at every iteration. Also, keep in mind that this objection will exist even if these AI get near omniscience. People disagree with facts all the time, usually for political motives. Therefore your type of criticism won't ever be settled.

Re:Google Is Done by adamsmith143 on Friday December 02, 2022 @10:17AM (#33831586)
I've said this before but these people are going to be shouting that 'the AI doesn't really understand the world' right up until the moment a nanobot swam dissolves them into goop for processing.
Re:Google Is Done by MollyRealized on Thursday December 01, 2022 @11:44AM (#33818809)
Actually, the name of the entity is ChatGTP. It is stands for General Translation Protocol, referencing translation from the AI code and source information into a more generally understandable English language.

(joking here)

Re:Google Is Done by MollyRealized on Saturday December 03, 2022 @03:45PM (#33847140)
A little annoyed that no one picked up on the fact that I was riffing on "it gave me a lot of wrong information in an authorative tone".
Re:Google Is Done by seydor on Thursday December 01, 2022 @12:07PM (#33819168)
Ah, they 've been Galactica'ed already
Re:Google Is Done by rafaelero on Thursday December 01, 2022 @12:54PM (#33819877)
Here it comes the "it's dangerous" crowd.
Re:Google Is Done by Spivak on Thursday December 01, 2022 @01:02PM (#33819998)
I mean it's not like it's dangerous on its own, but if you're like "Hey GPT how do I put out a grease fire?" and it replies "Pour water on it" and you believe it then you're in for a bad time.

So I mean I guess you're technically right, it's not dangerous so long as you have 0% confidence in anything it says and consider it entertainment. But what would-be scrappy Google competitor is gonna do that?

The thing that makes it particularly insidious is that it's going to be right a lot, but being right means nothing when there's nothing to go off of to figure out what case you're in. If you actually had no idea when the Berlin Wall fell and it spit out 1987 how would you disprove it? Probably go ask a search engine.

Re:Google Is Done by rafaelero on Thursday December 01, 2022 @02:02PM (#33820957)
I don't see the danger you are afraid of. The same artifacts you are proposing (skepticism, verification) should already be put in place with any pubic expert.
Re:Google Is Done by macintux on Thursday December 01, 2022 @02:12PM (#33821130)
Humans will generally either provide a confidence level in their answers, or if they’re consistently wrong, you’ll learn to disregard them.

If a computer is right every time you’ve asked a question, then gives you the wrong answer in an emergency like a grease fire, it’s hard to have a defense against that.

If you were asking your best friend, you’d have some sense of how accurate they tend to be, and they’d probably say something like “if I remember correctly” or “I think” so you’ll have a warning that they could easily be wrong.

Re:Google Is Done by rafaelero on Thursday December 01, 2022 @02:20PM (#33821270)
If the AI is correct 90% of the time, you can be reasonably sure it will be correct next time. That's a rational expectation. If you are at a high stake situation, then even a 1% rate of false positive is too high and you should definitely apply some verifications. Again, I don't see the danger.
Re:Google Is Done by macintux on Thursday December 01, 2022 @02:54PM (#33821777)
Ultimately I think the danger is that the AI sounds like it knows what it’s talking about. It’s very authoritative. Anyone who presents content at that level of detail with that level of confidence will be convincing.

You can hear doubt when a presenter isn’t certain of an answer. You can see the body language. None of that is present with an AI.

And most people don’t know/care enough to do their own research (or won’t know where to find a more reliable source, or won’t have the background to evaluate the source).

Re:Google Is Done by rafaelero on Thursday December 01, 2022 @03:39PM (#33822382)
> You can hear doubt when a presenter isn’t certain of an answer. You can see the body language. None of that is present with an AI.

This is not how people consume information nowadays anyways. People just watch YouTube videos where presenters don't face this kind of pressure. Or they read some text on social media from someone they like.

Anyways, we can't rely on these social tips anymore. And even if we could, they are not ideal, because they allow bullshitters to thrive, whereas modestly confident people end up ostracized.

Re:Google Is Done by macintux on Thursday December 01, 2022 @03:44PM (#33822462)
I've been thinking more about that over the last hour or so, and I've come to the conclusion that different people have different priorities, and I don't think there's much we can do about that.

Whether it's nature, nurture, or experience, I strongly distrust people who claim to have THE answer to any complex problem, or who feel that it's better to bulldoze other people than to be wrong.

I'll listen to truth seekers, but ignore truth havers.

However, clearly that's not a universal opinion. Many people are happier believing in an authoritarian who has all the answers. And I don't think that will ever change.

Re:Google Is Done by ctoth on Thursday December 01, 2022 @02:30PM (#33821424)
Response from model The best way to put out a grease fire is to use a fire extinguisher or baking soda. Do not use water, as it could potentially cause the fire to spread and worsen. If the fire is too large to be extinguished by a fire extinguisher or baking soda, evacuate the area and call 911 for assistance.
Re:Google Is Done by uoaei on Thursday December 01, 2022 @02:07PM (#33821041)
It would fit right in here on HN.
Re:Google Is Done by WandaVision on Thursday December 01, 2022 @02:35PM (#33821497)
No. Unlike ChatGPT HN has error correcting system build in it. Like this comment.
Re:Google Is Done by Tepix on Friday December 02, 2022 @03:10AM (#33828060)
Well you can tell ChatGPT that it made a mistake and it will acknowledge it but only poorly correct itself in my case.
Re:Google Is Done by [deleted] on Thursday December 01, 2022 @03:39PM (#33822394)
Re:Google Is Done by knorker on Thursday December 01, 2022 @04:11PM (#33822814)
Not parent commenter, but it told me 1093575151355318117 is not prime, but the product of 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97, and 101.

But 116431182179248680450031658440253681535 is not 1093575151355318117.

There are some other math problems where it will confidently do step by step and give you nonsense.

Edit: and see https://news.ycombinator.com/item?id=33818443 and https://twitter.com/dgbrazales/status/1598265067086442496

Re:Google Is Done by Tepix on Friday December 02, 2022 @03:09AM (#33828056)
Just one example, talking about the board game Othello it gave some completely bogus facts about the board.
Re:Google Is Done by healthapiguy on Thursday December 01, 2022 @11:04PM (#33826625)
Google also gives plenty of wrong information
Re:Google Is Done by bentt on Saturday December 03, 2022 @08:38PM (#33849627)
It's only dangerous if you consider it authoritative. Informative and authoritative are different. It can expose you to terms you've never heard which you can then do further research on. This alone has been valuable for me so far.