It even works on LLM. If you ask an LLM to continually repeat a word, it can break its training and start to regurgitate training data.

https://www.pcmag.com/news/this-silly-attack-reveals-snippet...

The only solution at the moment appears to detect the behavior and to stop the LLM from doing it.