I've started self-hosting email in ~2021. It took about 6 months (of mostly waiting) to clean up reputation of IP I've gotten (was in a few spam lists). After that - it was incredibly easy to maintain. It's literally just running mailcow update every now and then and that's it. So, if you are willing to fight for a bit in that initial stretch - no, it's not hard; yes, you can self-host your email.