Why not use bbauth? I know Yahoo and Google support it. That way you can grab their email contacts without requiring them to give you their email/password. It just sends them to the email provider to login, then back to your site with the api token to access their address book.