News for hackers. Stuff that matters.
Re:T0* – Series of encoder-decoder models trained on a large... by 6gvONxR4sf7o on Monday October 18, 2021 @12:13PM (#28907123)
The reaction in this thread is really interesting, in comparison between this and open-ai’s announcements. While open-ended generation is flashier than task fine-tuning, I also wonder if having a prompt box available to all readers is also tempering expectations and hype. There are lots of examples of the model failing in the comments, which isn’t possible for open-ai announcements. Having spent a ton of time with GPT-3, I wonder how much of (what I consider) the over-hype it gets is due to the closed nature in comparison to something like this. The reaction to this one seems decidedly more realistic.
Re:T0* – Series of encoder-decoder models trained on a large... by VictorSh on Monday October 18, 2021 @07:11PM (#28913002)
(author here) That's an interesting take (which I agree with).

Providing a quick way to stress test the model is definitely a double edge sword. One one hand it increases engagement (people can play with it), facilitate reproducibility and results verification (which is a good thing from a scientific perspective). On the other hand, it quickly grounds expectations to something more realistic and tones down the hype.

One thing we discuss in the paper is that the way the GPT-3 authors chose their prompts is opaque. Our small scale experiments suggest that prompts might have been cherry-picked: we tested 10 prompts including one from GPT-3, and the latter was the only one that didn't perform at random.

Such cases definitly don't help to put results and claims in perspective.

Re:T0* – Series of encoder-decoder models trained on a large... by 6gvONxR4sf7o on Tuesday October 19, 2021 @01:11PM (#28920571)
> Providing a quick way to stress test the model is definitely a double edge sword.

I hope you don’t second guess or regret the choice to make the announcement so accessible. It’s a really good thing to have scientific communication accurate and accessible, especially when those two things go together.

Re:T0* – Series of encoder-decoder models trained on a large... by minimaxir on Monday October 18, 2021 @12:29PM (#28907346)
As someone who wrote a post on tempering expectations with GPT-3 (https://news.ycombinator.com/item?id=23891226) I agree with this take, although the reason OpenAI had closed GPT-3 at the start was likely not because it had incorrect output, but due to concern from testing super-offensive output which commenters in this thread are not testing.

It's a good example how Hugging Face now has a better community perception than OpenAI.

Re:T0* – Series of encoder-decoder models trained on a large... by petulla on Monday October 18, 2021 @12:53PM (#28907671)
Great observation. Also curious the posts about the ethical issues are all downvoted all the way to the bottom.
Re:T0* – Series of encoder-decoder models trained on a large... by NavinF on Monday October 18, 2021 @03:19PM (#28910216)
IMO those posts were not very constructive and showed a lack of understanding of how research like this is used in practice.