It's problematic for the same reason other injection attacks are. It allows an attacker to get their "code" running in what should be a privileged environment you control. In this case, the "code" is a natural language prompt, but the principle applies.

Think about what would happen if an app scraped a web page into which someone has placed instructions for an LLM. The app then passes that instruction as context to the LLM, which is using the ReAct model [0] so it can run code, communicate with other apps, download content from the web etc. It's dangerous even if the LLM is just instructed to output JSON which the app uses for some other purpose.

Prompt injection is a fairly trivial problem if you're using an LLM to write a blog post, for example. But the deeper they are integrated into complex applications, the more dangerous it becomes.

[0]: https://arxiv.org/abs/2210.03629