The world isn’t ready for LLMs unleashed like this. Any plugin could do the same thing, and exfiltrate any part of the chat context with very little instruction engineering to bypass what things guardrails exist on the plugins side.
My conclusion is that with LLMs, the assumption always needs to be that the LLM might be capable of outputting any content, and the client needs to safely handle whatever is returned.
If this is confusing, control the server and have it return an image, or 404 instead of an image, doesn't matter, while logging the request.
You will find the exfiltrated data in the request log, and you will not find an image in the request log.
Of course, if your request handler is an image generator, you can generate an image containing your URL request. But by the time your generator gets the request, the data was leaked.
If remote images were blocked from markdown as they are in many other applications, the URL request would not be made by the browser. The mitigation is that ChatGPT should refuse to render image tags or make requests for images if given a 3rd-party URL. I'm not sure if we're talking past each other or what, but I'm not talking about whether the server returns an image. I'm saying, ChatGPT will request an image from an external domain as part of rendering markdown in the responses, and that can be used as an exfiltration method.
The response from the server has nothing to do with it. I'm talking about the fact that ChatGPT will request arbitrary urls without user input if the URLs are formatted in a markdown block as an image src.
Everyone always is because why on earth wouldn't this be blocked?
That wuzzie has contacted OpenAI about this multiple times and gotten companies like Microsoft to say, "you know what, 3rd-party images are things that nobody allows in most contexts and it shouldn't be enabled in places like Bing chat" and OpenAI in particular seems to have consistently decided that it's a core part of the chat capabilities for some reason -- it's on of the reasons why I say that OpenAI does not take security seriously.
It really should not be a debate whether or not remote images should be blocked from markdown. Your email client blocks remote images. Most forums either block remote images or proxy them. There is no reason at all for ChatGPT to have that the ability to display them.
Every company makes security mistakes, that's not a problem; it's when a company doubles down on security mistakes that their attitude about security becomes apparent.