> The clients doing SMTP themselves allows them to communicate sending failures to the user. How would you do that for an email just sitting in an IMAP folder?

I would introduce a folder for logs.