Sure, the security of your communications to someone depends on how well they protect them, not just you. That’s always true.

But most end-to-end encrypted apps aren't configured by most of their users to send their messages and encryption keys directly to the author of the app. iMessage is.