On top of that, sometimes warnings come from parts of the code you don't control, like a scanner generated by flex.
I think it's better to have a strong policy of not introducing warnings and fixing them whenever a particular compiler is foud to emit them, but having -Werror in the default Makefile can quickly inundate the project's mailing list/IRC channel with "doesn't compile" complaints from people with exotic compilers.
I think SQLite has a good attitude: http://www.sqlite.org/faq.html#q17
"Some people say that we should eliminate all warnings because benign warnings mask real warnings that might arise in future changes. This is true enough. But in reply, the developers observe that all warnings have already been fixed in the compilers used for SQLite development (various versions of GCC). Compiler warnings only arise from compilers that the developers do not use on a daily basis (Ex: MSVC)."
You are absolutely right that not all the warnings are accurate, and that not all code should be changed to satisfy a every compiler. But I'm arguing that there is a long-term benefit to choosing a single compiler, and making the modifications necessary to allow this compiler to run without warnings, even if the behaviour is provably correct without doing so.