News for hackers. Stuff that matters.

A comment on A look at Apple’s new Transformer-powered predictive text model. Parent comment

Re:A look at Apple’s new Transformer-powered predictive text... by dchest on Sunday September 17, 2023 @07:52AM (#37544055)
Yes, they are applied. Here's OpenAI doc which describes how to set various sampling parameters for GPT-3:

https://platform.openai.com/docs/api-reference/completions/c...

See presence_penalty and frequency_penalty.

Sampling techniques is one of important arts of LLMs, you'll can find a lot of papers on them.

In general, smaller are more prone to repetition, but you can get caught in it even with larger models.

[ Parent | on HN ]
Re:A look at Apple’s new Transformer-powered predictive text... by dchest on Sunday September 17, 2023 @11:15AM (#37545909)
To clarify: I meant that in general they are commonly applied, but in this case they weren't as the author confirmed. The repetition, of course, doesn't happen all the time.
[ Parent | on HN ]