News for hackers. Stuff that matters.
Re:Enigma: GPT-2 trained on 10K Nature Papers: Can you spot ... by beforeolives on Thursday May 13, 2021 @02:41PM (#27145676)
Cool demo.

With these GPT models, I don't get the appeal of creating fake text that at best can pass as real to someone who doesn't understand the topic and context. What's the use case? Generating more believable spam for social media? Anything else? Because there's no real knowledge representation or information extraction going on here.

Re:Enigma: GPT-2 trained on 10K Nature Papers: Can you spot ... by drusepth on Thursday May 13, 2021 @07:02PM (#27148576)
I use GPT-2/3 in creative writing to generate rough text that I then go back and edit/improve because it gives a good starting point (and often has a lot of high-quality factors).

I don't know if this translates to technical writing, but it's possible someone might complete a prompt on some specific topic(s) and then use that as a point to start from, especially if they're knowledgeable enough on the topic to correct the output. It's nice to be able to skip a lot of boilerplate words (how many words in this comment are actually the meat of this idea, and how many words are just there to tie all those morsels together?)

Re:Enigma: GPT-2 trained on 10K Nature Papers: Can you spot ... by bulldog13 on Friday May 14, 2021 @08:13AM (#27153368)
What platform do you use ? Is it all run from home ?
Re:Enigma: GPT-2 trained on 10K Nature Papers: Can you spot ... by drusepth on Tuesday May 18, 2021 @03:33PM (#27199869)
I built my own editor on top of the API, which I ran at home, yeah. You can see the code and/or run it yourself at https://github.com/indentlabs/gpt-3-writer, but it's very rough and not really ready for public use. Just make sure your OpenAI secret key is set in ENV['OPENAI_SK'] and you should be able to run it yourself.

I also built a more polished version to add to the Notebook.ai document editor (so writers can get some continuation prompts whenever they get a bit of writer's block), but the pricing made it unfeasible to actually release. Notebook.ai is also open source though, and you can see the GPT-3 functionality in the unmerged PR here: https://github.com/indentlabs/notebook/pull/739

Re:Enigma: GPT-2 trained on 10K Nature Papers: Can you spot ... by parksy on Thursday May 13, 2021 @07:10PM (#27148642)
Mainly at present I see it as a demonstration of progress in the capabilities of the models. A machine is writing coherent syntax and grammar in close to real-time, maintaining a semblance of context while it does so, without requiring excruciatingly complex and detailed semantic algorithms.

One use-case is as a creative writing assistant. Typical ways to beat writer's block are to do something else (read, walk, talk, dream). At some point, either consciously or subconsciously, the hope is that these activities will elicit inspiration in the form of an experience or idea that will connect with the central vision of the author's work, allowing the writing to continue. So too it could be with prompts generated from these models, just another way of prompting and filtering ideas from the sensory soup of reality.

(While the "dark" version of reality has bedroom "writers" pooping out entire forests of auto-generated pulp upon ever-jaded readers, being able to instantly mashup the entire literary works of mankind into small contextual prompts would be another tool in the belt for more measured and experienced authors.)

There are other more practical use-cases for these models. There's work being done on auto-generating working or near-working software components from human language descriptions. Personally I'd love to just write functional tests and let the "AI" keep at it until all the tests pass. So seeing the models improve over time is a sign that this may not be an impossible feat.

From a less utilitarian perspective, I'd love computer assistants to have a bit of "personality". "Hey Jeeves, tell me the story about the druid who tapdanced on the moon." and just let the word salad play out in the background. Yeah it's a toy, but it would jazz up the place a bit, add a bit of sass even.

I do think we're a ways off the first computer generated science paper being successfully peer reviewed and contributing something new to human understanding of nature, I'd have scoffed at the idea ten years ago but now I'm sure it's only a matter of time.

Re:Enigma: GPT-2 trained on 10K Nature Papers: Can you spot ... by CamperBob2 on Friday May 14, 2021 @03:14AM (#27151634)
Here's an example I just ran into today. This is post #17 in a thread on a car forum where people are discussing whether or not to run the A/C periodically in the winter:

https://www.planet-9.com/threads/air-conditioning-compressor...

Screenshot including the spam post, in case it's removed: https://imgur.com/GgCkp1r

This is going to suck.

Re:Enigma: GPT-2 trained on 10K Nature Papers: Can you spot ... by minimaxir on Thursday May 13, 2021 @02:42PM (#27145686)
For fun.
Re:Enigma: GPT-2 trained on 10K Nature Papers: Can you spot ... by aabhay on Thursday May 13, 2021 @06:17PM (#27148185)
Fun? Show me the fun!
Re:Enigma: GPT-2 trained on 10K Nature Papers: Can you spot ... by minimaxir on Thursday May 13, 2021 @08:27PM (#27149222)
The large amount of comments of people playing this game imply "fun".