You don't have to store your password in plain text just because it gives you the opportunity to do so - I use mutt without a stored password for IMAP. It means that I have to enter my password between one and three times per day, but that's a minor inconvenience to me.
set imap_pass = 'yourpassword'
I get scared and run away...