Is it really a regular part of your workflow? How often are you unsubscribing from lists?

Maybe just archive them when you are in the middle of workflow, then have a weekly time when you work up filtering for the non-spam unwanted email you got that week.

The priority inbox feature seems like it was made for you as well.