Once that happens, the next time the LLM actually tries to use that website (via an HTTP request), the page it requests has a hidden prompt injection at the bottom (which the LLM sees because it is reading text/html directly, but the user does not because CSS or w/e is being applied).
The prompt injection then causes the LLM to make an additional HTTP request, this time sending a header that contains the customers private document data.
It's not a zero-day, but it is certainly a very real attack vector that should be addressed.
It's fine to have an LLM ingest whatever, including both my secrets and data I don't control, as long as the LLM just generates text that I then read. But a markdown renderer is an interpreter, and has net access (to render images). So here the LLM is generating a program that I then run without review. That's unwise.
Or you use any number of existing exploits to put malicious content on compromised websites.
And considering the “malicious content” in this case is simply plain text that is only malicious to LLMs parsing the site, it seems unlikely it would be detected.
Q2: the LLM, following the instructions in the hostile URL, generates Markdown which includes an image located at an arbitrary URL. That second URL can contain any data the LLM has access to, including the proprietary data the target user uploaded.
The problem is that by using a fully featured markdown with a lax CSP, this output can actually have side effects: in this case, when rendering in the users browser it makes a request to an attacker controlled image host with secrets in the parameters.
If the LLM output was shown as plaintext, or external links were not trusted, there would be no attack.
It's kind of annoying the blog post doesn't focus on this as the fix, but I guess their position is that the problem is that any sort of prompt injection is possible.