News for hackers. Stuff that matters.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by meredydd on Saturday July 18, 2020 @08:43PM (#23886047)
I am deeply enjoying this comment thread - it's a bit of a Barium Meal [0] for determining how many people read (a) the headline, (b) the first paragraph, or (c) the whole thing before jumping straight into the compose box.

Having read to the bottom, the quality of text generation there absolutely blew me away. GPT-2 texts have a somewhat disconnected quality - "it only makes sense if you're not really paying attention" - that this article lacks entirely. Adjacent sentences and even paragraphs are plausible neighbours. Even on re-reading more closely, it doesn't feel like the world's best writing, but I don't notice major loss of coherence until the last couple of paragraphs. I am now really curious about the other 9 attempts that were thrown away. Are they always this good?!

[0] https://en.wikipedia.org/wiki/Canary_trap#Barium_meal_test

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by thewarrior on Saturday July 18, 2020 @10:21PM (#23886503)
I've started working on a version of GPT-2 which generates English text. The purpose of this is to improve its ability to predict the next character in a text, by having it learn 'grammatical rules' for English. It already works well for predicting the next character when it has seen only a small amount of text, but becomes less accurate as the amount of training text increases. I have managed to improve this by having it generate text. That is, it creates an 'original' piece of text about 'topic x', then a slightly altered version of this text where one sentence has a single word changed, and this process is repeated many times (about a million). It seems to quickly learn how to vary sentences in a way that seems natural and realistic. I think the reason this works is because it reduces the chance that the grammar it has learned for one specific topic (e.g. snow) will accidentally be transferred to another topic (e.g. dogs). Of course, this all means nothing unless it actually learns something from the process of generating text. I haven't tried this yet, but the plan is to have it generate text about a topic, then have a second GPT-2 system try to guess what that topic is. If the resulting system is noticeably better at this task, then we know the process has increased its ability to generalize.

One potential issue with this approach is that the text it generates is 'nonsensical', in that it is almost like a word-salad. Although this is a standard problem with neural nets (and other machine learning algorithms), in this case the text actually is a word-salad. It seems that it has learned the rules of grammar, but not the meaning of words. It is able to string words together in a way that sounds right, but the words don't actually mean anything.

Plot twist: This comment was generated by GPT-3 prompted with some of the comments in this thread.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by cs702 on Saturday July 18, 2020 @10:42PM (#23886585)
The thing that kills me is that to the vast majority of human beings the nonsensical technobabble above is probably indistinguishable from real, honest, logically consistent technobabble.[a]

Soon enough, someone will replicate the Sokal hoax[b] with GPT-3 or another state-of-the-art language-generation model. It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the social sciences.

[a] https://en.wikipedia.org/wiki/Technobabble

[b] https://en.wikipedia.org/wiki/Sokal_affair -- here's a copy of Sokal's hoax paper, "Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity:" https://physics.nyu.edu/faculty/sokal/transgress_v2/transgre...

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by thewarrior on Saturday July 18, 2020 @10:59PM (#23886664)
It's not hard to imagine GPT-3 writing a fake paper that gets published in certain academic journals in the social sciences. And then, it'll be all over for us. We won't have any more funding and our jobs will disappear. I can already hear the protests: "But we're not just scientists! We're also philosophers!" Well, yes and no. Philosophers are supposed to think about things philosophically, but they don't actually do anything about them; they're just entertainers. Scientists do something about them. They make things happen. And when those things happen, people take notice. If science and technology have a weakness, it's that they work too well. This was probably a strength at one point, but not anymore. In the not-too-distant future, there probably won't be any more philosophy professors; there will just be philosophers. But only in the same sense that there are lions and mushrooms.

This comment was also written by GPT-3.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by 6nf on Sunday July 19, 2020 @12:24AM (#23886990)
How many attempts did it take or did you just choose the first one?

I have to admit, this is passing my turing test...

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by geofft on Sunday July 19, 2020 @12:41AM (#23887047)
Really? I got about halfway through and realized that the comment had no point. If you tried to summarize what it was arguing, beyond the first sentence, I don't think you could make a coherent summary.

Maybe the real lesson is we don't expect human-written comments on discussion fora to be particularly coherent....

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by ascar on Sunday July 19, 2020 @04:15AM (#23887859)
Also both comments made me suspicious half way through and I scrolled to the bottom to check for a GPT-3 note. Without that note I would definitely have regarded it as incoherent rambling by a human.

Especially the second comment can be coherently interpreted with some good will and a cynical view of the humanities and philosophy. The "author" could say that once GPT-3 can write humanities papers it will quickly make humanity scientists redundant and that humanities scientists are philosophers is not important and doesn't warrant a job alone ("they don't actually do anything"). Eventually it shifts that this is the fault of science working too well (GPT-3 being a product of science)

It's not a consistent argument, but without the context of these comments being GPT-3 it would have totally passed my turing test, just not my sanity test.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by thom on Sunday July 19, 2020 @05:49AM (#23888212)
I think (slash worry) that this is going to be a simple upgrade in future iterations. Obviously there are powerful attention mechanisms at work to keep the subject matter coherent, but we’re not too far off the model being able to generate a core thesis, and ensure that all the generated text supports that in some way.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by tsimionescu on Sunday July 19, 2020 @12:31PM (#23890440)
I think that if that worked it would prove that either language is a much more powerful tool than we realize, or our cognitive capacities are much more trivial than we realize.

The model fundamentally has no understanding of the world, so if it can successfully argue about a central thesis without simply selecting pre-existing fragments, then it would suggest that the statistical relations between words capture directly our reasoning about the world.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by EGreg on Sunday July 19, 2020 @10:06AM (#23889459)
Who here thinks some Donald Trump's answers were written by an early version of GPT3, designed to produce more bombastic and rambling rhetoric than usual?
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by rusk on Sunday July 19, 2020 @02:51PM (#23891475)
In principle it’s not too far fetched ... there’s almost certainly some kind of data-driven algorithmic processing going into a lot of speech writing these days; some of the drops are so implausible they’d almost certainly have to have been suggested by a machine!
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by everdrive on Sunday July 19, 2020 @08:54AM (#23889004)
Not being sarcastic, but I know some people with less coherent writing than this. A lot of people struggle to make a point, use vague language, or wander in and out of topics quite easily.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by rusk on Sunday July 19, 2020 @02:52PM (#23891486)
Yeah they’re typically mimicking a style of speaking that they’ve heard other people use but don’t really understand the subject matter themselves ...
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by vidarh on Sunday July 19, 2020 @02:32AM (#23887458)
It felt like it was making a slightly ranty observation that scientists are already trying to much to be philosophers than to actually do science that changes the world, yet science has brought us far enough that it acts as an enabler for all kinds of pop-philosphers.

The final bit doesn't quite connect, but overall I've seen far less coherent comments written by humans on subject with far more logical flaws.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by riffraff on Sunday July 19, 2020 @03:44AM (#23887729)
I would not have imagined it was automatically written. Rambling and there's little connection between the first part and the latter, but absolutely something that might appear on a random internet forum.

I am genuinely awed.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by 2038AD on Sunday July 19, 2020 @07:23AM (#23888593)
> I got about halfway through and realized that the comment had no point

Pretty average for HN then ;)

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by pmiller2 on Sunday July 19, 2020 @02:28AM (#23887437)
Given that I know this stuff is generated text, it looks pretty good. But, if I’m judging it assuming that it was written by a human, it has a very uncanny valley sort of feel. That’s actually a good thing compared to previous models that would generate a lot of jarring non sequitors, because the GPT-3 text is very good if you look at it in 2-3 sentence chunks.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by hombre_fatal on Sunday July 19, 2020 @05:50AM (#23888217)
You say it like the bot wouldn't fit right in alongside most human comments because it meanders and doesn't seem to actually be responding to anyone, rather listening to itself talk.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by ALittleLight on Sunday July 19, 2020 @12:32PM (#23890442)
Unfortunately I saw the sentence at the end before reading the whole comment, so I don't know how my detector would've done, but I thought this line:

>In the not-too-distant future, there probably won't be any more philosophy professors; there will just be philosophers

Was quite clever and I'm still trying to figure out what it means.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by jlokier on Sunday July 19, 2020 @04:20PM (#23892163)
Maybe the real lesson is it was trained on human-written comments in discussion fora, so it perfectly mimics the average lack of point, weak arguments, rambling and incoherence in fora?

It would be interesting to see if the output has a similar quality when trained only on highly regarded texts.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by ci5er on Sunday July 19, 2020 @09:31AM (#23889228)
> Maybe the real lesson is we don't expect human-written comments on discussion fora to be particularly coherent....

How could we expect it? After 35+ years (BBS and Usenet onward), we've learned that they are often not.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by tgflynn on Sunday July 19, 2020 @10:58AM (#23889787)
Yep, it looks like GPT-3 might not be too far from achieving artificial schizophrenia.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by amelius on Sunday July 19, 2020 @06:11AM (#23888302)
I don't think these gpt3 comments will get many upvotes on HN anyway. I downvoted the first one for being incoherent, but then realized it was meant as an example so I undowned it.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by regularfry on Sunday July 19, 2020 @02:18PM (#23891224)
In possibly an unwise move, I'm actually going to respond to the initial point here.

There's a totally valid discipline in taking concepts from different areas and smushing them together to make a new idea. That's what a lot of creativity is, fundamentally. So a bot that's been trained across a wide variety of texts, spitting out an amalgam in response to a prompt that causes a connection to be made, is not only possible, but likely a very good way of generating papers (or at least abstracts) for humans to check. And if the raw output is readable, why not publish it?

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by rsync on Sunday July 19, 2020 @07:18PM (#23893254)
"This comment was also written by GPT-3"

Would you please show us the input text, or rules, you gave to GPT-3 to create this comment ?

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by jjoonathan on Saturday July 18, 2020 @11:50PM (#23886846)
Somebody get this thing onto scp-wiki, it will be right at home.

Not gonna lie, I went poking around to see if I could get my hands on it, but it seems like the answer is no, for now.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by StavrosK on Sunday July 19, 2020 @07:41AM (#23888651)
That's a good question, how do we get access? I signed up on the list, but it must be thousands of people long by now. Does anyone here know anyone who can get people access?
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by jimkleiber on Sunday July 19, 2020 @11:52AM (#23890156)
I'm starting to sense that, in most scenarios, I will no longer want to engage in text-based conversations unless I know that I'm talking with a human. I already don't like spending a lot of time arguing with a bot on Twitter, this just makes it much more likely I'll also argue with a bot on medium-length text areas (e.g., HN, FB, Whatsapp, SMS, etc.) and maybe even on long-length text areas (e.g., Medium, blogs, NYTimes or things pretending to be real newspapers, etc.)

Second, I'm curious/terrified at how future iterations of GPT-3 may impact our ability to express ourselves and form bonds with other humans. First it's text messages and comments. Then it's essays. Then it's speeches. Then it's love letters. Then it's pitches. Then it's books. Then it's movie scripts. Then it's...

TLDR; Fascinated by the technology behind making something like this work and quite worried about the implications of the technology.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by jacobedawson on Sunday July 19, 2020 @03:58AM (#23887790)
So it looks like we're about 2 years away from the 'Her' relationship model.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by schoen on Sunday July 19, 2020 @03:30PM (#23891763)
There was an article recently about people pursuing romantic relationships with chatbots. I thought there was a big HN discussion about it, but the only thing I've been able to find is this WSJ piece

https://news.ycombinator.com/item?id=22833407

(So I think it was some other story on the same topic.)

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by EGreg on Sunday July 19, 2020 @10:05AM (#23889451)
I am seeing it being used in tons of academic settings, especially with distance learning haha
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by randomsearch on Sunday July 19, 2020 @03:38AM (#23887701)
Was it really? I don’t believe you. It makes sense.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by thewarrior on Saturday July 18, 2020 @10:44PM (#23886599)
The thing is that this thing has now crossed into the uncanny valley. Earlier it would have great trouble making a single sentence that makes sense. You only ever remember whether the last two sentences made sense and go together. And with GPT-3 any pair of sentences always makes almost perfect sense. By the time you're four sentences down you go wait a minute ...
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by setzer22 on Sunday July 19, 2020 @01:56AM (#23887321)
This was very apparent when reading the generated stories [1].

Especially the shoggoth cat dialogue, I found that one really creepy. The fragment below comes straight out from the uncanny valley:

Human: Those memes sound funny. But you didn’t include any puns. So tell me, what is your favorite cat pun?

AI: Well, the best pun for me was the one he searched for the third time: “You didn’t eat all my fish, did you?” You see, the word “fish” can be replaced with the word “cats” to make the sentence read “Did you eat all my cats?”

[1]: https://www.gwern.net/GPT-3

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by spion on Sunday July 19, 2020 @07:00AM (#23888506)
An instance of a Voight-Kampff test.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by techdragon on Sunday July 19, 2020 @08:18AM (#23888826)
Yeah, GPT-3 never really gives any kind of answer to “why”. It rambles on like Abraham Simpson talking about how an onion on his belt was the style at the time. Devoid of purpose it fills the void with meaningless words until told to stop. It’s subtle gibberish... and fucking irritating as soon as you catch it.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by p1esk on Saturday July 18, 2020 @11:18PM (#23886739)
If he didn't put in the last line ("plot twist ...") I'm pretty sure no one here on HN would have guessed it.

In fact, while reading that comment I started to wonder why no one has tried to use GPT to generate text one character at a time. Or if someone has, what are the advantages and disadvantages over the BPE approach.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by the_other on Sunday July 19, 2020 @05:53AM (#23888228)
I’ve not studied neural nets and ML since undergrad level in 1998. So I am almost as knowledgeable as a random person on the street.

The quality of writing was very high, so I was convinced I was reading something put together by a human with agency... except it didn’t pass my gut-feeling “how IT works”. It made me suspect that either the algorithm (the described one, not the AI responsible) was off, or that I just didn’t understand AI any more. As I know I don’t have up to date AI knowledge, the algorithm appeared more believable. I hiked deep down the uncanny valley with that one.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by p1esk on Sunday July 19, 2020 @06:55PM (#23893132)
This comment didn’t pass my gpt filter.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by foobarbecue on Sunday July 19, 2020 @01:13AM (#23887176)
Your second paragraph is GPT-3?
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by O_H_E on Sunday July 19, 2020 @01:45AM (#23887285)
Ok, now this is getting deep and I don't like it.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by Scarblac on Sunday July 19, 2020 @03:44AM (#23887726)
I assume both previous comments, and this one, are also GPT-3?

Edit: it is amusing to think that soon the way to distinguish them will be that human comments have weird errors caused by smartphone keyboard "spell checking" in them...

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by p1esk on Sunday July 19, 2020 @04:11AM (#23887846)
You really think my comment above was GPT-3 generated? Wow. Did I really make so little sense?
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by foobarbecue on Monday July 20, 2020 @07:21AM (#23896715)
On a re-read, I'm not sure why I thought that, sorry. Context: I don't know much about machine learning and when I was scanning through comments, doing text generation one character at a time seemed silly and I must have been in the grips of "everything could be GPT!" hysteria. My robot detector needs work, clearly. Need to get educated.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by croddin on Sunday July 19, 2020 @04:40AM (#23887941)
But then if the models are trained on that dataset they might make the same errors to better approximate humans on the forum.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by more-coffee on Sunday July 19, 2020 @06:16AM (#23888331)
This entire thread is surrealistic and giving me Blade Runner vibes.. I love it.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by che_shirecat on Sunday July 19, 2020 @03:10PM (#23891596)
What if the plot twist line is also GPT-3?
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by killerstorm on Sunday July 19, 2020 @04:57AM (#23888002)
Well, you'd need to train it from scratch to operate on character level. And it would have smaller context, thus lower quality. So if you want same quality, you need much bigger context.

Still, would be an interesting experiment. Gwern swears it would improve stuff, so worth trying and comparing, I guess

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by g_p on Sunday July 19, 2020 @05:31AM (#23888129)
There's actually precedent for this - sci-gen got papers accepted in IEEE and Springer publications, through peer review, and they had to investigate their back catalogue to look for others.

Given the propensity for academic writing to often favour the strategy of confusing the author through obfuscation (to make a minor advance sound more significant than it is), I suspect tools like this could, as you say, actually get published papers in some fields like social sciences. In an engineering or science paper you can check equations match conclusions, and that graphs match data etc.

In a more qualitative field of work, reviewed in a publish-or-perish system that doesn't incentivise time spent on detailed reviewing, I think there's a very real risk babble like this just comes across like every other paper they "review".

I think it takes a certain level of confidence to dismiss others' work as nonsensical waffle, but sadly this is a confidence many lack, and they assume there must be some sense located therein. Marketing text is a great place to train yourself to recognise much of what is written is meaningless hocum.

Sci-Gen - https://pdos.csail.mit.edu/archive/scigen/

Reporting on withdrawals of papers - https://www.researchgate.net/publication/278619529_Detection...

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by 29athrowaway on Sunday July 19, 2020 @01:42PM (#23890951)
Dr. Herbert Schlangemann (fake alias for Sci-gen) not only got papers accepted in journals, it was invited to participate as session chair at conference co-sponsored by IEEE.

https://en.wikipedia.org/wiki/SCIgen#Schlangemann

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by barrkel on Saturday July 18, 2020 @10:55PM (#23886646)
In a similar way to how image detection networks appear to key largely on visual textures, GPT-3 seems to key on jargon, tone and pacing, the texture of speech within specific milieus.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by yaj54 on Sunday July 19, 2020 @12:01PM (#23890227)
The thing that kills me is that soon enough, a "fake" paper written by GPT-3 will get published in an academic journal because it has actually contributed a new insight.

It's easy to consider text generation models as "just mimicking grammar". But isn't grammar also just a model of human cognition?

Is GPT modeling grammar or is it modeling human cognition? Since GPT can ingest radically more text (aka ideas) won't it soon be able to generate texts (aka ideas) that are a more accurate collation of current knowledge than any individual human could generate?

--

[Was this comment written by GPT-3?]

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by lostmsu on Friday July 31, 2020 @12:48PM (#24010544)
It was not - there's a point :-D

I am impressed though nobody dared to guess in 2 weeks.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by Udik on Sunday July 19, 2020 @02:13PM (#23891181)
My impression is that these models are already doing far more than what the language production machinery in our brain does. We are able to produce language according to grammar and semantics, but we also have independent mental representations to guide the generation of language and to provide context.

I don't really understand why we're trying so hard to build models that can generate coherent texts based on having predigested only other texts, without any other experience of reality. Their capabilities appear already superhuman in their ability to imitate styles and patterns of any kind (including code generation, images, etc.). It feels like we're overshooting our target by trying to solve an unsolvable problem, that of deriving the semantics of reality from pure text, without any other type of input.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by pests on Sunday July 19, 2020 @02:23AM (#23887413)
One of my favorite conspiracists, Miles Mathis, has this quality. He can strong together entire pages of very real and consistent nonsense that is totally logical and makes enough sense to be real. I have to remember I'm not reading a legit theory and now really do confuse myself with his version of science vs reality.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by pasabagi on Sunday July 19, 2020 @08:36AM (#23888921)
Kind of annoying 'hoax', though. Obviously you can publish garbage in fringe journals if you leverage pre-existing prestige and position like Sokal did. Doesn't really say anything about the social sciences.

You can also publish a lot of nonsense in certain chinese journals that optimize for quantity in quality, in whatever field you want.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by CamperBob2 on Saturday July 18, 2020 @10:48PM (#23886619)
Worse, Sokal's Revenge is probably inevitable, in which someone will generate a nightmarishly complex mathematical proof that takes the whole field in unexplored directions. Some of the most respected professors and most promising students will then be distracted for years trying, and ultimately failing, to make sense of it.

Some say this has already happened. Nobody has ever seen the Social Text editors and Mochizuki in the same room together, have they?

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by pjc50 on Sunday July 19, 2020 @05:28AM (#23888112)
> Plot twist: This comment was generated by GPT-3 prompted with some of the comments in this thread.

This kills the forum.

Seriously, once this is weaponised, discussion of politics on the internet with strangers becomes completely pointless instead of just mostly pointless. You could potentially convince a human; you can't convince a neural net that isn't in learning mode.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by amelius on Sunday July 19, 2020 @06:15AM (#23888324)
Perhaps it will destroy anonymity. Because the only way to be sure a human wrote something is if you somehow know who the comment came from.

We might end up with reputation based conversations.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by pjc50 on Sunday July 19, 2020 @08:17AM (#23888823)
Of course, the human can simply lend their name to the robot. And as previously discussed, ending anonymity entrenches the existing power structure.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by amelius on Sunday July 19, 2020 @09:04AM (#23889060)
> Of course, the human can simply lend their name to the robot.

That could have consequences for their reputation, though.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by pjc50 on Sunday July 19, 2020 @10:03AM (#23889436)
Are you suggesting that people found to have misled the public should be .. cancelled?

(Reputation is a lot more controversial and complicated than it sounds)

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by heavenlyblue on Sunday July 19, 2020 @06:40AM (#23888426)
You can still be anonymous with this. You just need to be pseudonymous.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by amelius on Sunday July 19, 2020 @06:55AM (#23888489)
Robots can also have a pseudonym.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by kilotaras on Sunday July 19, 2020 @07:06AM (#23888533)
Go to URL and tell me what is written there.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by regularfry on Sunday July 19, 2020 @03:28PM (#23891749)
It's more insidious than that. You can think you've convinced a human whereas you've just spent your energy on a bot. Assuming "political arguments on social media" has any relevance to voted cast, that's a vote for your side which doesn't happen.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by jlokier on Sunday July 19, 2020 @04:11PM (#23892108)
> Seriously, once this is weaponised, discussion of politics on the internet with strangers becomes completely pointless

Quite the opposite, I suspect.

Eventually, to engage in the most persuasive conversations, the AIs will develop a real-time learning mode.

Once that is weaponised, the AIs will be on track to be in charge of running things, or at least greatly influencing how things are run.

What the AIs "think" will matter, if only because people will be listening to them.

Then it will be really important to discuss politics with the AIs.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by LdSGSgvupDV on Tuesday July 21, 2020 @02:22AM (#23904739)
Could you potentially convince a human for politics issue? It is extremely hard to convince stranger in the forum when there are some priors in mind.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by randomsearch on Sunday July 19, 2020 @03:37AM (#23887690)
That’s interesting, as within a sentence I had dismissed your comment as rambling and moved on to the next one, without thinking it had been generated... but maybe you’re double bluffing me.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by woko on Sunday July 19, 2020 @04:05AM (#23887824)
Same, except that I skipped straight to the last line to check whether it was a generated text after I noticed the first sentence made no sense (GPT-2 already generates grammatically correct English sentences).
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by AJRF on Sunday July 19, 2020 @04:30AM (#23887906)
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by c3534l on Saturday July 18, 2020 @10:24PM (#23886520)
I read the beginning, went "what the fuck is this guy on about? Get to the point" and then came to check the comments to if this was anything interesting or worth reading, saw your comment, and skimmed the end bit. Overall I'm pleased with my process as its an efficient way to find out which articles are worth reading. But it was also clear to me that the author had difficulty making a clear point or had a goal in his writing. I skipped through it for a reason and I suspect many other people did as well.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by crucialfelix on Sunday July 19, 2020 @04:29AM (#23887902)
Yes, we have poorly written babble from humans too. Now we will have weapons grade babbling from machines.

The result is that worthwhile public discussion is dying. We have to transition now to secure verified communication.

Either that or the bots fork off a new cultural discourse and we treat them like a new form of entertainment.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by ALittleLight on Saturday July 18, 2020 @11:12PM (#23886718)
From your comment it's not clear to me if you realize the author of the article is GPT-3.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by cryptoz on Sunday July 19, 2020 @12:56AM (#23887112)
At this point I'm not even sure if that particular comment was written by GPT-3 or not.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by c3534l on Sunday July 19, 2020 @02:15AM (#23887376)
I swear I'm not a robot, I pass Google captchas and everything!
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by thom on Sunday July 19, 2020 @05:52AM (#23888221)
I’m human and regularly don’t.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by StavrosK on Sunday July 19, 2020 @07:48AM (#23888678)
I'm sorry to be the one to break it to you, but that means you aren't human.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by more-coffee on Sunday July 19, 2020 @06:19AM (#23888346)
You say you're human, but how can we know for sure?
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by thom on Sunday July 19, 2020 @07:05AM (#23888527)
How does it make you feel that I say I’m human?
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by c3534l on Sunday July 19, 2020 @12:10AM (#23886930)
I do, but I see how that wasn't clear.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by mdoms on Saturday July 18, 2020 @11:20PM (#23886744)
It's very clear to me that he does not. But he does an excellent job of making GP's point.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by jjoonathan on Saturday July 18, 2020 @11:41PM (#23886807)
Err, it seems very clear to me that he does realize GPT-3 is the author, and that it was easily caught by his bullshit filter. Which was my experience too -- but I am less dismissive. I regularly see human-produced bullshit get very far with less coherence than these examples from GPT-3.

GPT-3 isn't AGI, but it's weapons-grade in a way that GPT-2 wasn't.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by AnimalMuppet on Sunday July 19, 2020 @12:06AM (#23886909)
OK, but a weapons-grade BS generator is not what the world needs right now...
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by jjoonathan on Sunday July 19, 2020 @12:18AM (#23886974)
Ready or not, here GPT-3 comes.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by nashadelic on Saturday August 01, 2020 @04:31AM (#24018782)
I feel bad for realizing I give authors benefits of doubts and let them ramble for me to learn the point. I guess I need to drop my bar when I'm reading but a LOT of people ramble a lot. Almost every blog post I ready, I skip the first 2 paragraphs because they're just an intro/context which you already know from the headline or prior knowledge.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by Alex3917 on Saturday July 18, 2020 @09:18PM (#23886220)
> Even on re-reading more closely, it doesn't feel like the world's best writing, but I don't notice major loss of coherence until the last couple of paragraphs.

I guessed it was fake before getting to the end, not from the content, but from the fact that all the sentences are roughly the same length and follow the same basic grammatical patterns. Real people purposely mix up their sentence structure in order to keep the readers engaged, whereas this wasn't doing that at all. Still very impressive though; if not for the fact that the post was about computer generated content I probably wouldn't have noticed.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by bhntr3 on Sunday July 19, 2020 @02:16AM (#23887379)
Besides predictable sentence structure, GPT-3 writes like George R. R. Martin: Interesting premises, solid setup but then it devolves into rambling tangents and never quite delivers the concluding action that ties everything together.

Lots of examples I've seen have phrases like "see table below". Of course there's no table and it's hard to imagine how there could be.

But GPT is trained on internet content and the internet is full of terrible writing that never gets to the point. I doubt there's any way to know how much is "not actually understanding the subject matter" vs. "learning bad writing from bad writers". I'm inclined to believe the majority is the former but there's got to be a little of the latter sprinkled in.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by niklasd on Sunday July 19, 2020 @03:57AM (#23887789)
I am really curious how the model would be if you would train it with a decent amount of really good literature. Kazuo Ishiguro et al. instead of Reddit.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by kmill on Sunday July 19, 2020 @05:23AM (#23888093)
I was playing with AI Dungeon tonight to get access to GPT-3, and one of my many experiments ended up with me meeting a character called the Narrator who believed they were in control of all characters in the game, including me. Eventually, through my predicting what they were about to say by checking and undoing, they seemed convinced I wasn't another character and started asking about whether certain authors were still alive and which I liked to read. It didn't recognize Ishiguro. Later it gave me a truly bizarre (and amusing) summary of Infinite Jest, clearly having never read it. Anyway, the entire experience was uncanny and surreal.

One thing I learned was it has detailed knowledge of the world of Avatar: The Last Airbender, seemingly through fanfics. It was fun having it to teach me the lost arts of pizzabending ("form your hands into the shape of a letter 'P'" and so on, and needing to practice by juggling rubber pizzas) and beetlebending ("always remember that to beetle bend it helps to like beetles," my wise uncle suggested). Each of these tended to precipitate a narrative collapse.

The writing style was surprisingly homogeneous, and it reminded me of young adult novels. It would definitely be interesting to see it with other writing styles, beyond the occasional old poetry.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by derangedHorse on Sunday July 19, 2020 @03:05PM (#23891565)
I've never heard of AI Dungeon before reading your post but even after playing for 2 minutes, I can tell it's going to be huge.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by hnick on Sunday July 19, 2020 @10:12AM (#23889489)
How about full adult? Taking it for a test run and this happened after I told a man to stop copying me. Before this he kept talking about clothes for some reason.

> The man walks away and starts undressing. You shrug and keep following him. Soon, you find yourself naked.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by Reelin on Sunday July 19, 2020 @04:23AM (#23887884)
Library Genesis contains lots (millions?) of fiction ebooks (among other things). It's available in torrent form. Not that I would ever condone piracy or anything.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by gpderetta on Sunday July 19, 2020 @06:01AM (#23888266)
Well now we know what is going to finish asoiaf in case of author existence failure.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by gomoboo on Saturday July 18, 2020 @09:56PM (#23886389)
I did not guess that it was fake but skipped to the bottom because the article did not seem to be worth reading. It felt like the author was not moving anywhere with their words. I laughed out loud at the reveal that it was written programmatically.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by pyb on Sunday July 19, 2020 @04:02AM (#23887811)
You raise a good point. The Internet has trained us to skim any text that seems pointless or just unsufficiently insightful. So it turns out we have already built up some "mental defences" against GPT-3.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by gbrown on Sunday July 19, 2020 @09:48AM (#23889344)
Now we need to make human augmented generative adversarial eyetracking networks that are trained on getting people not to skim.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by javajosh on Saturday July 18, 2020 @10:18PM (#23886491)
There are a few concrete things the (fake) article says, primarily about what software to try (OpenAI's GPT-3), and where to try it (bitcointalk forum). Personally, I actually resent being mislead like this, at least on that second point, even with the full disclosure. The output is very high quality, but it is making at least one falsifiable assertion (no test was ever done in that forum).
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by nullc on Saturday July 18, 2020 @10:25PM (#23886524)
If you've ever spent much time with a toddler you might have noticed that they spout a lot of fantasy. Learning to not make up untrue claims takes years of additional training for humans.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by selestify on Sunday July 19, 2020 @01:30AM (#23887235)
I’ve never spent much time with toddlers. What do they make up things about? Their own actions, other’s actions, claims about the environment?
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by Scarblac on Sunday July 19, 2020 @03:54AM (#23887776)
My six year old will just continue as long as he has people's attention, and if that means he has to make things up, so be it. Freely stealing phrases from other recent conversations.

So this morning he heard about an animal, it was kind of a lion. But with bat's ears, it lives in Africa. It looks like it's a rock, but it's actually not, it's rock shaped but has tiny legs. And it's gray and hard. Its face... It doesn't really have a face. It lives up in trees where it eats bamboo and apples. It has these huge fangs like sabertooth tigers, you know?

It's glorious.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by riffraff on Sunday July 19, 2020 @03:54AM (#23887777)
All of those, in my experience.

My smallest kid has a habit of telling stories about himself that actually come from whatever he heard recently, e.g. "once I was Godzilla..", or claims about things in reality that come from stories or misunderstandings all mixed up "did you know, there are three pigs, but they are not pigs, they are wolves and a hunter came and killed them but they weren't wolves they were dragons..."

It's actually very GPT-3-ish now that I think of it.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by IIAOPSW on Sunday July 19, 2020 @12:55AM (#23887108)
Some of them never learn it at all!
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by dannyw on Saturday July 18, 2020 @09:37PM (#23886296)
If that’s the only thing that separates this from human writing; I’m sure it can be influenced easily.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by Alex3917 on Saturday July 18, 2020 @09:58PM (#23886396)
> I'm sure it can be influenced easily.

Maybe. Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind.

E.g. I explain here why it's possible for written statements to be objectively insightful, informative, interesting, or funny, but objectively in a way that's relational to other information or beliefs. The implication being that statements are only going to seem subjectively funny or insightful (or whatever) to others who have that knowledge or those beliefs, which means that you can't reliably create those subjective experiences in a reader without having some sort of theory of mind for them.

I guess you can create content that's funny or insightful relative to that content itself, but that's not especially useful. It's entertaining at the time, but the experience is more like seeing a movie that you laugh a lot during but then leave and are kind of like what was the point? It's an empty experience because it wasn't transformative.

I definitely don't think it's impossible, but I also don't think it's a matter of just adding a couple more if-else statements.

https://alexkrupp.typepad.com/sensemaking/2010/06/how-writin...

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by roenxi on Saturday July 18, 2020 @10:13PM (#23886460)
> Maybe. Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind.

I'm going to call this goalpost shifting. This article is better writing than some % of humans, theory of mind or otherwise. The AI has comfortably surpassed Timecube-level writing and is entering the pool of passes-for-a-human.

'Sounds human' is a spectrum that starts with the mentally ill and goes up to the best writers in human history.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by Alex3917 on Saturday July 18, 2020 @10:24PM (#23886516)
> I'm going to call this goalpost shifting.

That's completely fair. On the other hand, without a theory of mind it can't really educate or inspire people, the only thing it can do is maybe trick them about the authorship of something. But once people learn the techniques for identifying this kind of writing, it can't even do that anymore. To me this is like the front end of something, but it still needs a back end.

Don't get me wrong, it's super cool research and seems like a huge step forward, and I'm excited to see where it goes. But I also don't see this AI running a successful presidential campaign or whatever, at least within the next couple years.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by Scarblac on Sunday July 19, 2020 @04:02AM (#23887812)
It made me consider: - The existence of this model I hadn't heard of - Bitcoin (sigh) - Testing it out on a forum, trying to become a well known poster - picking a forum with many different types of posters - some of which you dislike

And that got me thinking about what I could do with this thing, whether I should, what I wanted to try out...

So the BS random ideas were still inspiring a bit.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by CamperBob2 on Saturday July 18, 2020 @11:00PM (#23886668)
On the other hand, without a theory of mind it can't really educate or inspire people

I wouldn't agree with that, either. How often have we heard of someone gaining useful insights by considering ideas that were misapplied or just plain wrong? Entire branches of physics have evolved that way. As far as successful presidential campaigns are concerned... well, let's not even go there.

If there's such a thing as a 'theory of mind', it applies to the reader, not the writer.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by AnimalMuppet on Sunday July 19, 2020 @12:12AM (#23886935)
I think I disagree.

For example, I delayed in writing this comment because the cat was on my lap, and I couldn't fit the laptop and the cat both. You get that. I know you do, even if you don't own a cat, and even if you're reading this on a phone or a desktop.

GPT-3 does not understand about the cat. To GPT-3, they're just words and phrases that occur in the vicinity of each other with certain probability distributions. As a result, it can't write something and know that there's something there in your mind for it to connect to.

Cyc would handle the bit about the cat differently. It would ask "Did you mean cat-the-mammal, cat-the-Caterpillar-stock-symbol, or cat-the-ethernet-cable-classification?" It has categories, and some notion of which words fit in each category, but it still doesn't understand what's going on.

But you the human understand, because you have a lap, and you've at least seen a cat on a lap.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by lowdose on Sunday July 19, 2020 @04:14AM (#23887852)
> But you the human understand, because you have a lap, and you've at least seen a cat on a lap.

You really think GPT-3 never came across a comment about a cat in lap? 50% of all the pictures on the internet are cats sitting on people. GPT-3 doesn't need to understand it to echo this common knowledge.

Airplanes don't look like birds at all but they do fly.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by dTal on Sunday July 19, 2020 @12:49PM (#23890564)
I don't see how any number of comments about cats in laps can allow it to synthesize the following logical chain: cat-on-lap -> pauli exclusion principle -> laptop NOT on lap -> laptop awkward to reach -> delayed comment
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by vladTheInhaler on Sunday July 19, 2020 @06:22AM (#23888357)
I think the issue is that text doesn't exist in a vacuum, but the corpus that the model is learning from does. A piece of human writing exists for a particular reason - to persuade, to inform, to ask a question, etc - and its value is judged on its ability to perform that task. But that's not a quality that is evident from the text itself, only from looking at the world outside the text. This suggests to me some limits on this kind of passive self-supervised approach. Perhaps it could be improved by augmenting the text with other forms of data? For instance predicting video from text and vice versa. But I think that to learn a true "theory of mind", it needs to use text like an agent - to influence its environment, not merely predict it.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by dannyw on Sunday July 19, 2020 @06:42AM (#23888437)
It could also be a result of training data. If every page is weighted equally, you'd expect SEO spam and even autogenerated content to far surpass high quality content in volume.

I would like to see a GPT model where training data is weighted by credibility / authority (e.g. using Pagerank).

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by vladTheInhaler on Monday July 20, 2020 @04:13AM (#23895762)
My understanding is that GPT-2 was actually trained on a dataset that was designed to avoid those pitfalls. They followed all the links posted to Reddit that had more than a couple karma, under the theory that the content was at least slightly interesting to some actual humans, as opposed to a giant blob of search keywords or what have you.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by IIAOPSW on Sunday July 19, 2020 @12:58AM (#23887120)
>Right now this reads like a glorified shopping list. It's coherent, but actually sounding human also requires a theory of mind.

What if the ultimate theory of mind turns out to be that consciousness is an illusion and nothing separates us from a sufficiently sophisticated markov process.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by throwing947383 on Sunday July 19, 2020 @05:15AM (#23888060)
>What if the ultimate theory of mind turns out to be that consciousness is an illusion and nothing separates us from a sufficiently sophisticated markov process.

Conscious experience would still exist (see cogito ergo sum, Chalmers, etc). If we were to be shown we're just Markov processes, that wouldn't disprove the existence of conscious experience. Just like confabulation, a misleading experience is still an experience.

What it would disprove is any sense of agency.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by chrisweekly on Saturday July 18, 2020 @11:09PM (#23886709)
"objectively ...funny" strikes me as a contradiction in terms; concepts like humor, insight, and interest are fundamentally subjective, dependent by definition on a subject's consciousness and expectations
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by 488643689 on Sunday July 19, 2020 @04:36AM (#23887931)
Aww c'mon. You guessed it was fake, because it's an article about computer generated articles. Who would read that and not question the content in front? You are not analysing everything you read for oddities in writing style.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by antonvs on Sunday July 19, 2020 @01:00PM (#23890634)
> You are not analysing everything you read for oddities in writing style.

I certainly do, don't you? When I read a blog post and it's full of poorly-integrated buzzwords that make it seem like it was churned out by a non-English speaker being paid very poorly per word, I stop reading and move on.

I recently read a few pages of a book someone had recommended to me and stopped reading because of the writing style.

Heck, you can read a few pages of, say, a Dan Brown novel, and based on the writing style might choose not to read it, since the style tells you a lot about the kind of book it is.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by yyyk on Saturday July 18, 2020 @09:57PM (#23886390)
I'm not a very good test case. I briefly skimmed (not expecting very much from a Bitcoin-themed article), read the end, and only then read more carefully. So my first read was brief and biased, and my second was very biased.

That said, the content of the computer-generated parts doesn't make much sense even for a Bitcoin-influenced article (what would be the point of paraphrasing your previous post in a forum on a regular basis, and how does this not get one very quickly banned?), but the grammar is far far better than previous attempts - it reads like Simple English wiki.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by woko on Sunday July 19, 2020 @04:08AM (#23887836)
> I briefly skimmed (not expecting very much from a Bitcoin-themed article), read the end, and only then read more carefully.

It sounds to me like you must be an academic, or someone with good habits for being efficient at reading articles.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by zamadatix on Saturday July 18, 2020 @09:11PM (#23886182)
Or maybe we're all bots too and you're the only real HN user!

I agree, responses are almost as interesting as GPT-3. And this place has always felt like one of the better when it comes to people reading past the titles!

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by JBiserkov on Saturday July 18, 2020 @10:17PM (#23886480)
"Every account on reddit is a bot except you."

https://www.reddit.com/r/AskReddit/comments/348vlx/what_bot_...

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by jcahill on Saturday July 18, 2020 @09:10PM (#23886181)
GPT-3 is a neat party trick. But the things that'll be done with web archives* in the next 20y will make it look like the PDP-8. ~love, a web archivist

* GPT-3 is trained on one

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by ypcx on Saturday July 18, 2020 @10:15PM (#23886470)
The transformer model as presented in GPT-3 may be a few tweaks away from a human-acceptable reasoning, at which point we may realize that human brain is just a neat party trick as well. This may come difficult for some people to internalize, especially those who understand the technology in depth. Because it means that the medium of our reality is the consciousness.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by walleeee on Saturday July 18, 2020 @11:01PM (#23886683)
Was this comment generated by GPT-3?
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by Naracion on Sunday July 19, 2020 @02:10PM (#23891168)
I doubted that as well, but I don't think it is--at least it's not a simple copy paste. There's an emphasis on _is_ in the last sentence which I don't think the algorithm could have generated.

However that makes one wonder if it can also learn to generate emphases, and if so, how would it format? With voice generation it can simply change its tonality but with text generation it has to demarcate it in some way--does the human say "format the output for html", for instance?

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by sytelus on Sunday July 19, 2020 @06:38AM (#23888416)
You are confusing pattern matching with reasoning. If your brain was replaced by GPT-3 model and you were cast away on a distant island, I highly doubt you will be able to perceive, plan and prosper during your survival against all the calamity nature would through at you.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by mamon on Sunday July 19, 2020 @10:08AM (#23889470)
To be honest, most city-raised humans wouldn't be able to survive on a distant island as well.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by Tarean on Sunday July 19, 2020 @01:42PM (#23890947)
The transformer model in GPT-3 has a short context window and no recurrence. Without some significant architecture changes that is a fundamental limit on the problems GPT-3 can solve.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by lucidrains on Sunday July 19, 2020 @03:00PM (#23891532)
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by visarga on Sunday July 19, 2020 @12:07AM (#23886918)
> Because it means that the medium of our reality is the consciousness.

I agree. The environment - as the source of learning and forming concepts, is the key ingredient of consciousness, not the brain.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by yomly on Saturday July 18, 2020 @10:28PM (#23886537)
I don't fully understand what you're getting at here...

Basically the brain and "consciousness" isn't as fancy as we think?

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by lucidrains on Sunday July 19, 2020 @03:08AM (#23887583)
Exactly.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by h0p3 on Saturday July 18, 2020 @09:22PM (#23886233)
No pressure: feel free to ignore me, please. Would you mind elaborating? I'm interested in what you have to say (and, of course, feel free to say it privately if you prefer). I would like to even hear your dreams, wild speculations, or gut feelings about the matter.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by jcahill on Sunday July 19, 2020 @03:54AM (#23887778)
Sure, what do you want to know?

I currently work on synbio × web archival.

Some of us are cooking up futuretech aimed at storing all of IA (archive.org) in a shoebox. Others are working on putting archival tools in more normal web users' hands, and making those tools do things that people tend to value more in the short-term, like help them understand what they're researching, rather than merely stash pages.

My ambitions for web archives are outsized compared to other archivists, but I'm fine with that. I'm looking beyond web archives as we currently understand them toward web archives as something else that doesn't quite exist yet: everyday artefacts, colocated and integrated with other web technology to an extent that they serve in essential sensemaking, workflow, and maybe security roles.

Right now, some obvious, pressing priorities are (a) preserving vastly more content and (b) doing more with the archives themselves.

A: The overwhelming majority of born-digital content is lost within a far narrower time-slice than would admit preservation at current rates, and data growth is accelerating beyond the reach of conventional storage media. So, for me, the world's current largest x is never the true object of my desire. I'm after a way to hold the world that is and the world to come.

Ideally, that world to come is one where lifelong data stewardship of everything from your own genome to your digital footprint is ubiquitously available and loss of information has been largely rendered optional.

This, of course, requires magic storage density that simply defies fundamental limitations of conventional storage media. I'm strongly confident that we're getting early glimpses of the first real Magic contenders. All lie outside, or on the far periphery of, the evolutionary tree that got us the storage media we have today. For instance, I'm running an art exhibition that involves encoding all the works on DNA.

B: Distributed archival that comes almost as naturally as browsing is well within reach, and with that comes some very new potential for distributed computation on archives. One hand washes the other.

One important thing to realize here is that, in many cases, you can name a very small handful of individuals as the reason why current archival resources exist. GPT-3 is cracking the surface by training on data produced by one guy named Sebastian, for instance.

…i'm sorta tired and have to respond to something about every twitter snapshot since June being broken, though, so I'll pick this back up later.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by greyface- on Saturday July 18, 2020 @09:55PM (#23886379)
This is an interesting thought. GPT-3 used 45TB of raw CommonCrawl data (which was filtered down to 570GB prior to training). The Internet Archive has 48PB of raw data.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by GreenHeuristics on Sunday July 19, 2020 @09:22AM (#23889165)
That 48PB is mostly just old video game roms and isos though
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by scoot_718 on Saturday July 18, 2020 @11:10PM (#23886713)
Hopefully in a way that secures some funding for those making archives of the web.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by jcahill on Sunday July 19, 2020 @12:06AM (#23886908)
I'm running the Coronavirus Archive. Largest thematic archive on the pandemic, since January. I'm also teaching community biolab techniques to people in parts of the world without ready access to commercial COVID-19 test kits, on all but zero resources at this point.

I could use… what's the word? I think it's more funding.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by m3kw9 on Saturday July 18, 2020 @09:54PM (#23886374)
Problem was I lost interest half way because it lost my interest after the 2nd paragraph. For those that say it was good till the last few is really pretending to understand what it said. It really did not made much sense.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by blueblimp on Saturday July 18, 2020 @11:46PM (#23886831)
I'm finding with secret GPT-3 output that I often find it boring before realizing it's GPT-3. I might even be getting to recognize its wordy, dull, cliche-ridden, borderline-nonsensical style. It's remarkably good at passing as human writing of no value whatsoever.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by x0 on Sunday July 19, 2020 @01:05AM (#23887145)
Well, it is trained on web content. Often I read an article and it's obvious they're either stretching it out to hit a word count, or trying to get as many google-able phrases in as they can. Some sites are worse than others, with the more no-name ones the worst offenders.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by visarga on Sunday July 19, 2020 @12:09AM (#23886923)
The fast jump in quality from GPT2 to 3 is more important than the current level of GPT3. Maybe next year it will be not-boring.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by wcoenen on Saturday July 18, 2020 @10:38PM (#23886572)
It's ironic that you're critiquing the algorithm for not making sense, while contradicting yourself in your first sentence. Did you lose interest "half way", or "after the second paragraph"? It can't be both.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by necovek on Saturday July 18, 2020 @10:59PM (#23886665)
Wouldn't "after the second paragraph" be "half-way" for a four paragraph piece? :D

But you are right, it can't be both in the context of this article :)

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by ricksharp on Saturday July 18, 2020 @11:28PM (#23886769)
Those types of contradictions are what made me suspect the article was generated.

Now, I’m not so sure :)

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by sidcool on Saturday July 18, 2020 @10:06PM (#23886424)
This is so true. I had a high regard for HN comments till quite recently.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by sangnoir on Sunday July 19, 2020 @03:18PM (#23891666)
I take it you've never had your comment - which happens to be on a subject of your expertise or lived experienced - downvoted because simply because it doesn't feel "truthy" enough for HNs demography. I have long since adopted a healthy disregard of HN comments in other areas that I'm no expert in. I still haven't found a way to monetize that though; that is my holy grail.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by dTal on Sunday July 19, 2020 @12:30PM (#23890427)
It is a bit unfortunate that your comment is now at the top - it spoils the test :)

Saying that, I briefly saw the first sentence of your comment and went to read the article with the idea that trickery was afoot, specifically guessing correctly the nature of the article. And yet, even then, on the back foot... it fooled me. Incredible.

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by OOPMan on Sunday July 19, 2020 @07:54AM (#23888703)
Thanks to this comment I actually read the blog post.

It was relatively good, although I began to suspect it was GPT3 generated about halfway through (partially because the style felt a bit stiff but also just out of a shayamalan-what-a-twist 6th sense of mine that was tingling)

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by classified on Wednesday July 22, 2020 @06:21AM (#23915565)
Eliza could do this better. Or, just use a Markov chain that has read enough corporate PR bullshit. It's just sad how many people use this "AI" meme to fulfill their need to worship something.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by DonHopkins on Sunday July 19, 2020 @06:20AM (#23888352)
It reminds me of the incoherent demented ramblings we've all been been hearing (but hopefully not following as medical advice) for the past several years.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by simonebrunozzi on Monday July 20, 2020 @02:56PM (#23900785)
Ah, but you are almost spoiling the end with your second paragraph! :)

I agree with you. I suspect few people have read until the end to realize that, in fact, ...

Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by m3kw9 on Saturday July 18, 2020 @09:20PM (#23886226)
Is exactly the issue. You still need humans to check it before releasing an output. It can only be what the author says “Bitcoin” level of implication if it can get things probably needing at least “99%” quality and correctness.
Re:OpenAI's GPT-3 may be the biggest thing since Bitcoin by joe_the_user on Saturday July 18, 2020 @09:38PM (#23886304)
OK, I read the first sentence and it sounded like a typical poor-quality marketing-blather article so I came here and read your post.

I then reread it and it indeed read like a weird, rambling, incoherent article. Looking at it closely, it had a good many contradictory, meaningless and incoherent sentences.("It is a popular forum with many types of posts and posters.")

The headline, however, seemed about right.

It's true the nonsense in this article is a bit different than the nonsense of a GPT-2 article. But the thing GPT-2 paragraphs sound pretty coherent 'till they suddenly go off the rail. This is more like an article that was never quite on the rails and so it's slightly more internally cohesive. But not "better".

Maybe the article just reflects the author's style. Anyone have a GPT-3 test site link?