Even large language models with billions of parameters get caught in the cycle. You don't usually see it exposed to users because there are sampling tricks applied, such as repetition/frequency penalty.

Such penalties would only exist for fine-tuned models. Base models have only the temperature setting. As the example at the end shows, even GPT-2 seems "smarter" than the Apple model, probably because of the number of parameters.

There's no such thing as "base models have only the temperature setting". Models do not have sampling settings (temperature, repetition penalty, etc), the sampling code does, which obviously you can use on any model.

For example, here's a function from llama.cpp that applies repetition penalty: https://github.com/ggerganov/llama.cpp/blob/master/llama.cpp...

Here's the one from transformers: https://github.com/huggingface/transformers/blob/0a55d9f7376...

To summarize how they work: you keep some number of previously generated tokens, and once you get logits that you want to sample a new token from, you find the logits for existing tokens and multiply them by a penalty, thus lowering the probability of the corresponding tokens.

I don't think such penalties were applied to GPT-2 or even GPT-3, yet they weren't repetitive like that.

Yes, they are applied. Here's OpenAI doc which describes how to set various sampling parameters for GPT-3:

https://platform.openai.com/docs/api-reference/completions/c...

See presence_penalty and frequency_penalty.

Sampling techniques is one of important arts of LLMs, you'll can find a lot of papers on them.

In general, smaller are more prone to repetition, but you can get caught in it even with larger models.

To clarify: I meant that in general they are commonly applied, but in this case they weren't as the author confirmed. The repetition, of course, doesn't happen all the time.

Yes, you're right, I should have mentioned it in the post, but I used pure greedy sampling for the GPT-2 outputs since I couldn't do anything but that for the Apple model. So temperature was set to zero, and there was no repetition penalty.

I don’t agree, it’s not very common for LLMs to get stuck in loops simply because loops are not commonly observed in the datasets.

Try running llama.cpp with 0 temperature and without repetition penalty and you'll sometimes get caught in a loop.

Or, if you're okay with a smaller LLM, go here, set temperature to zero and enjoy repetition: https://transformer.huggingface.co/doc/distil-gpt2

https://imgur.com/a/hF1Ewk4

That’s where you’re wrong. Raw LLM very often get stuck in loops, usually extremely small loops. There’s a big chunk of infrastructure that exists on the output end of any production LLM that exists explicitly for the purpose of preventing loops.

That post-processing infrastructure can use all kinds of mechanisms to prevent loops and induce more useful output. With the most basic simply systems simply refusing to select any output token that already in the input, to more complex stochastic process that explore the tree of possible outputs, to find branches whose overall result is improved by choosing less optimal immediate steps.

The vast majority of what make Chat GPT different to simpler GPT-3 models is this complex post-processing phase that allows designers to push and pull on the behaviour of the pre-baked static model underlying the chat interface.

Agreed 100%, and some of the "simple" approaches like repetition penalty, can harm output, because for example markdown tables repeat a lot of characters, and if you just blindly apply repeat penalty, it will just stop doing what you've asked for.

My guess is that they have some kind of state-aware sampler, and they know if they are in the table, etc. Because then you can sample in a much better way. Just like with grammars, but grammar itself is probably not enough.

Sampling and tokenization are IMHO the biggest open problems. We have something which kind of works but it's nowhere close to be perfect.

It is very common for small models, and sometimes even 7B model gets stuck.

There was an article/paper showing that GPTs (whole family) quickly get confident in looping if there is anything loop-like in the window. So basically, as soon as it loops once, it will never get out of it.

Note that sometimes loops are desired, like with docstring in the code, always starting before the function definition, and other structural things.