This has been oft-repeated conventional wisdom from years, but I think we're giving most spammers way too much credit.
I run several mail servers, and addresses that have not existed for well over a decade, and always give 5XX responses, still get spam.
As long as spam is profitable, because it pushes all negative externalities off to someone else, it is not worth a spammer's time to cull their lists.
I simply do not see this happening.
The reasoning being that it could be used by spammers to confirm an email address is real/valid after a user attempts to unsubscribe at which point they could sign them up to more spam. So don't expect your own emails to show the button by just adding the List-Unsubscribe header, unless you're using something like Amazon SES.