Fastmail has employees that have to be paid. You're paying that money instead of the time required to maintain your own email server.

I understand that, but paying 100x what a VPS would cost, that's a rip-off. (100x because a VPS can handle the email of 100 people or even more.)

But what does the price of a VPS have to do with anything? I doubt hosting would be their biggest cost.

Why shouldn't hosting be their biggest cost?

Well, think about the VPS you're proposing. Two euros a month is 24€ a year. Even if you're only paying yourself 6€ an hour, I'm skeptical that running an email server for 100 people would require less than four hours per year.