Are you on Mountain Lion? I couldn't use Mail.app on Lion as it was incredibly slow. The I realized it wasn't Mail.app's fault, but GPGMail's, and now that GPGMail has stopped working in ML my Mail.app feels as snappy as ever.