“spam wouldn’t scale” - unconvinced on this: spammers already mostly use other people’s compromised machines to do the sending; there is no cost to them here.

If you charge some cost per mail (whether that's CPU time or actual money), users/teams would check their spend and optimize accordingly. They'd notice runaway spend and act on it. The only reason why mail servers become compromised and nobody notices is that bandwidth for mail is generally too cheap to meter. On any 10s of megabits connection, sending a deluge of spam is trivial.