I know about the Host header, my thought was more about something like a Redirected-From header which would be set based on things like the host the <img src> or XHR came from. A simple sanity check on that (anything other than localhost is blocked) should suffice.