> people use the spam button as a lazy man's unsubscribe

This is the small mail server crux right here. If you’re a small mail server and a few of your emails have been spam binned instead of unsubscribed, it would likely lead to your whole server getting shit canned.