It’s how the original image recognition obfuscation techniques worked. You’d update an image by backpropagating from the desired output category.

LLMs are much more complicated structures than imagenets, so I’m sure there are a lot of challenges involved.

Also I imagine the input (output?) might be complete ghibberish.