Some numbers mentioned on the x32abi page hint on anywhere between 4% and 40% performance gains, if that is true then I'd think the benefits would outweigh the hassle of another architecture.
(Edit: Most middleware also ship straight-C versions of the routines; whether or not an x32 C compiler can measure up to handcrafted x86 or x86_64 assembly I don't know - but I'm guessing the much higher register count would help a lot. Regarding proprietary software: There are a great many server configurations that do not need anything beyond the standard open source packages available in Debian)
Could the cpu have a mode/flag where all pointers in cpu registers were treated as 32 bits (high 32 bits ignored)?
Alignment issues might make this impossible (loading a 32 bit address in a compatible way from memory/cache...) but it would be neat if it could work. For instance, if memory layout in 32 bit chunks were A... B..., there would have to be a way to load both A and B into the low 32 bits of a register.
But it's also yet another architecture. It's not binary compatible with either i686 or x86_64. You need your whole userspace compiled to use it. Middleware with embedded assembly (there's a surprising amount of this in glibc, and of course things like ffmpeg, libjpeg, etc...) needs to be ported. You can't run 32 bit binaries from proprietary sources.
And frankly the benefit over straight x86_64 is quite modest. I don't see x32 taking off. It's just not worth the hassle.