The vulnerability is Facebook's social contract with non-technical users, user trust based on mental models of how it should work. For users of privacy settings, trust matters.
A user knows things they are sharing can be used by those they share with. But when they delete something, in their mental model it's been deleted, and they become (with fair reason) upset if they learn it's not.
If users delete a Facebook post or change its privacy setting, it's gone or inaccessible. If they delete a photo, it's still there and still accessible. That's unexpected behavior.
There's no good non-technical reason text content and image content shouldn't be equally carefully managed by Facebook.
In the other case, users can use less well-documented features of their browsers to violate Facebook users' mental model of how much control they have over their photos.
The fact that Facebook could address the latter problem but hasn't doesn't upset me, because it's mooted by the former problem.
I acknowledge the PR problem they have blundered into by accepting this otherwise insignificant risk, though: bored tech journalists can use it to gin up sensational stories that further the narrative about Facebook's cavalier attitude towards privacy. I'm not suggesting that it was a good call on Facebook's part to do this.
Facebook could reduce its attack surface meaningfully by eliminating the static-file CDN. But this isn't a reasonable step to take; it generates minimal (infinitessimal!) privacy advantages for users while drastically complicating their service.
To your point, yes, these URLs remain in email archives and IM logs.
This is why Flickr changes an image's URL if you change its privacy settings.
The only thing that can happen as a result of the fbcdn cache is that a malicious friend could publish the stale fbcdn URL. But that same malicious friend could also publish the picture itself on any of a dozen photo sharing sites, and that "attack" is far more damaging: Facebook can't trace it or shut it off.
It is, I suppose, worth pointing out that this is a reason to be irritated at any friend who republishes your photos by hotlinking to the Facebook CDN.