Another reason to avoid -Werror is if you pull in third party code that you share your build options with. Even large stable open source projects regularly spew hundreds of warnings when you aggressively turn them on.

I'll generally build my own code with all warnings, but use as close as I can to the package maintainer's build settings for their code. Even then, you can get warnings because the third party developer doesn't use your exact compiler.