Depends, if you're able to poison the DNS of the mailprovider / hack the recipient mailserver or do a phising attack.

I just want to be able to communicate without sharing my phone number (since my phone number is bound to Swedish "Swish") meaning someone can get my ID from my phone number here.

This is why drug dealers use Wickr, Threema and others, because they don't expose identity, not because they're "safer".

I have a contact on Threema who I've met many times, but I have no idea how to contact him outside of Threema, because I don't know his identity and we'd both like to keep it that way.