Hi, a dev here. </s> means "end of sequence" for LLMs. If a model generates it, it forgets everything and continues with an unrelated random text. So I don't think that malicious actors are involved in this case.

Apparently, the Colab code snippet is just too simplified and does not handle </s> correctly. This is not the case with the full chatbot app at https://chat.petals.dev - you can try it instead or take a look at its code.

Thanks for the reply. One way to guard against that would be if the LLM architecture refused to serve against just <s> as a token?

And if this is possible, code changing attacks are possible.

Base llama has lots of lurid in it already.