For the type of vector arithmetic you do in games, SSE (much less later flavors like SSE2/3/etc.) is a big win. SIMD instructions for number crunching is huge.
Your example there is more GCC being shitty than anything else.
And angersock is missing the point: you can't take scalar code and rebuild it into SIMD (except in the very limited, never-works-as-well-as-you-think-it-should auto vectorization features in modern compilers), the parallelism needs to be designed in. That's not possible here without a rewrite of the game engine.
Scalar code is not trivially fixed by using SSE, true, but the majority of really obnoxious math being done (skinning, vector arithmetic, etc.) should be really easy to make really fast.
SSE wouldn't be a flags issue. Building a vectorized SSE library in "debug mode" would still produce vector instructions.