If I understand this correctly, you need to be managing your own DNS (not letting Fastmail do it), and you probably need to set p=reject in your DMARC so that non-fastmail servers can't spoof your addresses.

And if Fastmail allows Fastmail user A to spoof Fastmail user B, then the above still only protects you against non-Fastmail customers.