It takes at least five seconds out of my life. Sure, not much. But certainly more than the 0.5s it takes to hit the keystroke to mark as spam.

And that five seconds is a lowball estimate if the unsubscribe link is actually to a "manage your email notifications settings" page as is so common these days.