Thinking about it, it seems questions generally allow for some flexibility of interpretation reducing potential cause for offense, where statements allow far less latitude. The problem is that some questions require foundational understandings that may be offensive, or can really only be understood with malintent. "When did you stop beating your wife?" comes to mind. The question doesn't even allow for the possibility that you don't beat your wife.

Similarly, "Can I sleep with your wife" implies that the asker thinks it's a possibility worth inquiring about. So I basically agree with you on a practical level -- most people would not respond well to the underlying assumption. The question is a veiled statement ("I think you might be open to letting me sleep with your wife, so here I am asking"). In that sense, the question and the statement aren't very different, as you noted.