The difficulty is using the established methods to build out a whole system that is workable in realistic conditions.

A system that handle sending messages when one party is off-line. A system that doesn't require up-front key agreement between all parties. A system that handles moving devices somewhat gracefully. A system that gives decent guarantees on the identity of your counter party, without being very onerous.

Making a system that meets the above requirements and others I missed, and that remains as usable as if the encryption weren't there is hard.