Cross-domain messaging is easy, as long as you either accept all servers, or at least whitelist the ones you want, and have the S2S port forwarded, or better yet, no NAT at all. Same goes for file transfer, but I will admit I've never personally succeeded at a file transfer, but I couldn't get DCC to work either.
One of the big differences between this approach and email is that it would be a prompt-based system, instead of a switch applications and click the link. By prompt-based I mean that it would pop up a window asking, instead of you having to go look for it (and possibly open Outlook, Mail, or some other heavy app thats not running 24/7 like most people leave IM messengers)
As a set of protocols, SMTP and IMAP are both pretty well-suited for the task they set out to accomplish. Furthermore, the delays you mentioned in message delivery are largely due to intermediate spam filters and store-and-forward queues, not inherent issues with email as a transfer mechanism.