In the end, if you want a career in IT, or find it interesting, I'd say go for it. I've often thought about building my own end to end open-source mail service aimed at ease of administration. If I was ever rich enough to not have to work, that's one of the things I'd probably do after a few months off.
I'd suggest starting with a secondary domain and using Mail-In-A-Box on a VPS or smaller Cloud host. It's a decent starting point, and there are many other options. One of my favorites is SmarterMail (commercial, windows only though). I also ran a BBS for a number of years using Synchronet, which does email/pop/smtop and even nntp for group messages. I did have it configured with SendGrid at the time for outbound for a while, which handled delivery issues for me.
You will have headaches if you take it seriously and/or use it as a primary service without using a delivery service like MailGun or SendGrid, and even then you probably will have other headaches. My point isn't really do discourage so much as let people have a more realistic understanding. The issues the OP has are real. However, there is so much junk from bad actors that the well is poison.