News for hackers. Stuff that matters.
Re:GPTZero case study discovers it's only accurate on less t... by panarky on Sunday February 19, 2023 @11:09AM (#34858756)
As millions of people interact with ChatGPT, their writing will subtly, gradually, begin to mimic its style. As future versions of the model are trained on this new text, both human and AI styles will converge until any difference between the two are infinitesimal.
Re:GPTZero case study discovers it's only accurate on less t... by brookst on Sunday February 19, 2023 @11:16AM (#34858835)
One of the big complaints with LLMs is the confident hallucination of incorrect facts, like software APIs that don’t exist.

But the way I see it, if ChatGPT thinks the Python list object should have a .is_sorted() property, that’s a pretty good indication that maybe it should.

I work in PM (giant company, not Python), and one of these days my self-control will fail me and I will open a bug for “product does not support full API as specified by ChatGPT”.

Re:GPTZero case study discovers it's only accurate on less t... by howlin on Sunday February 19, 2023 @11:41AM (#34859127)
> LLMs is the confident hallucination of incorrect facts

This is a very common feature of delirium in people. Chatting with an LLM seems a lot like what it would be to talk to a clever person with encyclopedic knowledge, who is just waking up from anaesthesia or is sleep talking.

Re:GPTZero case study discovers it's only accurate on less t... by moffkalast on Sunday February 19, 2023 @02:59PM (#34861226)
Or just the average person on reddit's r/confidentlyincorrect.
Re:GPTZero case study discovers it's only accurate on less t... by matthewdgreen on Sunday February 19, 2023 @11:26AM (#34858955)
ChatGPT also thinks certain coding algorithms should violate well-known information-theoretic bounds ;) I’ll put a ticket in with Claude Shannon.
Re:GPTZero case study discovers it's only accurate on less t... by brookst on Sunday February 19, 2023 @11:32AM (#34859024)
I’m sure he’ll be excited to learn about this gap!
Re:GPTZero case study discovers it's only accurate on less t... by xiphias2 on Sunday February 19, 2023 @11:24AM (#34858923)
Or rather you can just catch method missing in the runtime and patch it with a chatgpt call
Re:GPTZero case study discovers it's only accurate on less t... by brookst on Sunday February 19, 2023 @11:34AM (#34859047)
Love it. Get ChatGPT to write the missing method, execute it this once, then store it in a file, update the current source file with the include to cache it for next time.
Re:GPTZero case study discovers it's only accurate on less t... by int_19h on Sunday February 19, 2023 @06:01PM (#34862792)
I can't find it, but someone already did a Python module that plugs into GPT-3 and automatically generates functions on the fly as you call them - and then the same for methods on the returned values etc.
Re:GPTZero case study discovers it's only accurate on less t... by dr_dshiv on Sunday February 19, 2023 @11:58AM (#34859351)
> it, if ChatGPT thinks the Python list object should have a .is_sorted() property, that’s a pretty good indication that maybe it should.

Yes! And when it hallucinates references for articles, often times those articles probably should exist…

Re:GPTZero case study discovers it's only accurate on less t... by visarga on Sunday February 19, 2023 @12:39PM (#34859815)
And if they don't exist, you ask the model to write them from title and link.
Re:GPTZero case study discovers it's only accurate on less t... by moffkalast on Sunday February 19, 2023 @05:06PM (#34862336)
The year is 2145.

When a new person is born their entire life is hallucinated in its entirety by the all great and powerful GPT. Deviation from His plan is met with swift and severe consequences.

Re:GPTZero case study discovers it's only accurate on less t... by visarga on Sunday February 19, 2023 @12:38PM (#34859799)
> if ChatGPT thinks the Python list object should have a .is_sorted() property, that’s a pretty good indication that maybe it should.

Hahaha, Python language fixing itself!!!

Re:GPTZero case study discovers it's only accurate on less t... by wankle on Sunday February 19, 2023 @11:40AM (#34859110)
AI will turn half the people into Eloi (https://en.wikipedia.org/wiki/Eloi) while the rest of us will become the Morlocks.
Re:GPTZero case study discovers it's only accurate on less t... by onos on Sunday February 19, 2023 @11:12AM (#34858780)
Interesting idea, but isn’t there variance in the output? Eg I’ve seen people ask it to “write in the style of x” etc and different people also clearly have different writing styles.
Re:GPTZero case study discovers it's only accurate on less t... by Filligree on Sunday February 19, 2023 @11:28AM (#34858979)
You can try, but it isn’t very good at that. The style remains very ChatGPT.
Re:GPTZero case study discovers it's only accurate on less t... by moralestapia on Sunday February 19, 2023 @11:15AM (#34858817)
+1

We train AIs but they also train us.

Re:GPTZero case study discovers it's only accurate on less t... by ominous on Sunday February 19, 2023 @12:13PM (#34859528)
Some related idea, in case you like to see that thought explore: https://medium.com/@freddavis/we-shape-our-tools-and-thereaf...
Re:GPTZero case study discovers it's only accurate on less t... by moralestapia on Sunday February 19, 2023 @08:24PM (#34863871)
Great read, thanks for sharing!
Re:GPTZero case study discovers it's only accurate on less t... by andyjohnson0 on Sunday February 19, 2023 @11:50AM (#34859233)
Prediction #1: Once enough ChatGPT output gets posted online, it will inevitably find its way into the training corpus. When that happens, ChatGPT becomes stateful and develops episodic memory.

Prediction #2: As more people discuss ChatGPT online, by late 2023 discussion of Roko's Basilisk exceeds discussion of ChatGPT. (half /s)

Re:GPTZero case study discovers it's only accurate on less t... by ClumsyPilot on Sunday February 19, 2023 @11:55AM (#34859299)
Or. ChatGPT will overtrain on it's own data and go to shit the way google search did
Re:GPTZero case study discovers it's only accurate on less t... by visarga on Sunday February 19, 2023 @12:44PM (#34859869)
Training on its own data is a tradition already. For example RLHF example pairs rated by humans are generated by the model. So even our best models trained on their own outputs + rating from human labellers. The internet is a huge rating machine, AI will distill this signal and improve even while ingesting its own text.
Re:GPTZero case study discovers it's only accurate on less t... by andyjohnson0 on Sunday February 19, 2023 @04:53PM (#34862240)
Meta-ChatGPT's loss function optimises for ChatGPT generating training data that maximises the shittyness of Google's LLM.
Re:GPTZero case study discovers it's only accurate on less t... by LelouBil on Sunday February 19, 2023 @08:42PM (#34864012)
Did you see the new Bing chat ?

#1 is already happening !

See here (other HN thread) : https://twitter.com/tobyordoxford/status/1627414519784910849

Re:GPTZero case study discovers it's only accurate on less t... by fullshark on Sunday February 19, 2023 @11:57AM (#34859328)
Sounds accurate and horrifying, I don't get the enthusiasm for this at all beyond a desire to be there first and make a ton of money. All manuscripts get a run through an AI editor, all business writing is even more soullessly devoid of purpose beyond accomplishing task X, all blogposts are finetuned for maximum engagement and therefore ad/referral revenue.

That's already happening I know but it will be amplified to the point that all humanity in writing in lost. All ideas in writing will be a copy of a copy of a copy and merely resemble something once meaningful. Time to go touch grass.

Re:GPTZero case study discovers it's only accurate on less t... by visarga on Sunday February 19, 2023 @12:47PM (#34859908)
Too much AI for you? You can fix your problem with even more AI! Get your own AI, running on your hardware, loyal only to you. It will act like a firewall between you, the vulnerable, hackable human, and the wild internet full of other AIs. To go out on the internet without an AI is like going for a walk during COVID without a mask, or browsing without AdBlock. Their AI will talk to your AI, and that's how you can be safe.
Re:GPTZero case study discovers it's only accurate on less t... by zdragnar on Sunday February 19, 2023 @12:18PM (#34859582)
So long as ChatGPT is forbidden from communicating in certain ways (swearing, speaking ill or positive of controversial people or topics, etc), convergence will never happen. People interact with other people more than they do ChatGPT, so the majority force will remain dominant.
Re:GPTZero case study discovers it's only accurate on less t... by joe_the_user on Sunday February 19, 2023 @03:48PM (#34861717)
An opposite possibility is that the commonness of ChatGPT will cause people to adopt a style as distinct from it as possible.

Of course, this might mean future Chatbots would successfully emulate that. But it's not impossible an "adversarial style" exists - this wouldn't be impossible to emulate but it might be more likely to cause the emulator to say things the reader can immediately tell are false.

One idea is to "flirt" with all things that people have come up with that AI chokes on. "Back when the golden gate bridge was carried across Egypt..."