News for hackers. Stuff that matters.
Re:GPT-3: Language Models Are Few-Shot Learners by wyattpeak on Friday May 29, 2020 @01:56AM (#23346680)
I don't know if it says something about text generation or human text processing, but whenever I read an example of computer generated text, all through I think "I can't tell this is machine generated, it seems completely natural," and the only giveaway is that at the end I have no idea what it said.

It's a pretty eerie feeling. It's as though both the AI and my short-term processing only pay attention to a context of a few sentences, so nothing seems off until I try to understand it as a whole.

EDIT: Thinking more, what it feels like most of all is reading a page of a book and not taking it in.

Re:GPT-3: Language Models Are Few-Shot Learners by theon144 on Friday May 29, 2020 @03:06AM (#23347057)
Reminds me of a relatively old article I read, "Humans Who Are Not Concentrating Are Not General Intelligences": https://srconstantin.wordpress.com/2019/02/25/humans-who-are...
Re:GPT-3: Language Models Are Few-Shot Learners by pasquinelli on Friday May 29, 2020 @02:42AM (#23346926)
To me it reads like a child telling a story, but that this child has an adult's ability to use language. When children tell a story they aren't going anywhere with it but don't know how to cover it up.
Re:GPT-3: Language Models Are Few-Shot Learners by carlmr on Friday May 29, 2020 @06:25AM (#23348298)
I know plenty of adults that can't seem to get to a point.
Re:GPT-3: Language Models Are Few-Shot Learners by lowdose on Friday May 29, 2020 @10:29AM (#23350314)
Kevin Hart on the Joe Rogan show comes to mind.
Re:GPT-3: Language Models Are Few-Shot Learners by tarsinge on Friday May 29, 2020 @07:54AM (#23348878)
Maybe someone drunk then?
Re:GPT-3: Language Models Are Few-Shot Learners by Nursie on Friday May 29, 2020 @04:10AM (#23347503)
> the only giveaway is that at the end I have no idea what it said

So it's a lot like corporate executive speak then?

I agree with your point, it does seem very much like valid speech, but somehow the informational content is missing. It's like speech without the actual comminication part.

Re:GPT-3: Language Models Are Few-Shot Learners by fnord123 on Friday May 29, 2020 @05:28AM (#23347984)
I've read a lot of academic papers like this.
Re:GPT-3: Language Models Are Few-Shot Learners by tarsinge on Friday May 29, 2020 @07:45AM (#23348804)
This has some parallels to generic random corporate PR or marketing speak. Some communications are already so automated and dehumanized that we are used to random content signed by a pseudo real person that we gloss over, and are more easily fall for GPT like generated content that has similar form. Edit: I mean I guess someone of a previous generation used to only read the newspaper and letters would more instantly spot something is wrong.
Re:GPT-3: Language Models Are Few-Shot Learners by wiz21c on Friday May 29, 2020 @03:23AM (#23347152)
That's really well put. And it does reflect how that kind of document is produced. So I'm not surprised at all.

Problem arises when you read that document without really trying to understand it. In that case, it might be enough to trigger some thoughts.

It also appears to me that two way communications, that is, social interactions, will be the only way to form truth. That is, AI produced content could erode some more the trust we put in of newspaper, TV news, etc (all forms of one way communication). Not that we've waited AI to distrust those, but well, on e more nail in the coffin :-)

(this text, although rather unclear, was written by a genuine human :-) )

Re:GPT-3: Language Models Are Few-Shot Learners by xamdam on Friday May 29, 2020 @09:14AM (#23349558)
Re:GPT-3: Language Models Are Few-Shot Learners by maest on Friday May 29, 2020 @05:30AM (#23347995)
That would suggest combining 2 models: one to decide what the macro text structure should be and a different one (GPT) to decide how to fill in all the text flesh.
Re:GPT-3: Language Models Are Few-Shot Learners by runawaybottle on Friday May 29, 2020 @09:17AM (#23349600)
That takes the whole write drunk, edit sober to a whole new level.

GPT2 the drunken novelist.

Re:GPT-3: Language Models Are Few-Shot Learners by urlwolf on Friday May 29, 2020 @07:37AM (#23348752)
This happens to me when I'm reading in a language I'm not very good at (German). Each sentence may make sense, but overall I feel I didn't get the point. I guess it's a cummulative error situation, where you reach a threshold after which the point is lost.
Re:GPT-3: Language Models Are Few-Shot Learners by Udik on Friday May 29, 2020 @08:17AM (#23349061)
The example above is impressive because it actually makes sense, except for the last sentence of the first paragraph: "But this time, his publicist is saying he'll be wearing a tux no matter what."

Remove that, and there is a typical if completely uninteresting celebrity argument: one has to play eccentric in public occasions (he did it last year, he's planning to do it again) and the other chides him for what she feels it's maybe a lack of respect? And he replies that despite his best intentions he can't go against his conscience. There, done. It's a perfect little piece ready to be served in some celebrity gossip magazine.

Re:GPT-3: Language Models Are Few-Shot Learners by tripzilch on Sunday May 31, 2020 @02:45PM (#23371611)
It sounds like someone talking without knowing (or caring) where they're going with it.

I think that's what missing. Usually we communicate with a certain goal in mind, to bring across some point. This text was generated without such a goal, you notice it doesn't really know when to stop talking. I wonder what was the stopping criterion, but I'm sure it wasn't "keep talking until all the information we want to convey has been mentioned".

Re:GPT-3: Language Models Are Few-Shot Learners by methyl on Friday May 29, 2020 @02:23AM (#23346813)
I have similar feelings. Human-written text in general has one coherent flow of ideas.

So what if we used text generation algorithms on a paragraph basis, so that the idea flow is still figured out by a human (input would be just an outline)? That should make the generated text feel like a whole, especially if we could preserve the style its written across the whole text.

Re:GPT-3: Language Models Are Few-Shot Learners by CardenB on Friday May 29, 2020 @06:08AM (#23348206)
This is because our current methods of machine learning are really good at finding patterns but not very good at enforcing underlying models.
Re:GPT-3: Language Models Are Few-Shot Learners by attilakun on Wednesday June 10, 2020 @09:04PM (#23484156)
Read Blindsight by Peter Watts. It touches on the phenomena that you are describing.
Re:GPT-3: Language Models Are Few-Shot Learners by andromeduck on Friday May 29, 2020 @02:58AM (#23347004)
Aka corp speak.