If you measure something, people will adapt their behavior to what you're measuring. If you measure productivity by volume of mails, people will try to increase the volume of mails at the expense of what is in those mails.
From that perspective, I think dashboards are useful as a sort of warning signal for things going wrong, but not as indicators of whether things are going right. So, the right way to approach it in my opinion is to measure whether mails are being sent at all, because that would indicate a major issue.
We use it on our board to track the volume of support e-mails going out the door for each support agent. It's very useful!