I'm running a rooted phone at the moment but you're essentially reducing the security of the phone. I wish I didn't have to root my phone to get all the features I want (removing apps, backing up apps to somewhere I control, redirecting audio)

I'm a fellow rooter also concerned about device security. I think full disk encryption, with the unlock key encrypted with a passphrase not stored on the device gives me more than the security that was lost by rooting my phone. Even recovery can't touch the OS or my files without me feeding it the passphrase.