The instruction set available is often not the most relevant thing. In many cases, the optimal code will differ wildly between CPUs regardless of the available instruction sets. Let's look at some quick examples of performance characteristics:
Athlon 64: Slow SSE2 unit, MMX is often faster than SSE2 for many functions.
Phenom: Very fast SSE2 unit, but missing SSSE3 support, so can't take use the same optimized functions in many cases as the Core 2 and above. Significantly higher instruction latency than Core 2, so needs significantly more pipelining.
Core 2 Conroe: SSE shuffle operations (punpck, etc) are excruciatingly slow: 4/2 for shuffles that use both arguments as input data (e.g. punpckldq), 2/2 for those that use only one argument (e.g. pshufb). Cacheline-split loads are extraordinarily painful (equivalent to an L1 cache miss, or ~14 cycles).
Core 2 Penryn: Same as Conroe, but shuffles are fast now (1/1 for everything basically).
Nehalem: Cacheline-split loads are cheap now (2 cycles), and shuffles have doubled throughput compared to Penryn (1/0.5).
This doesn't even get into the more instruction-specific messiness, like how some CPUs like movddup and movhlps on integer data while others don't. It's not uncommon to have 3 or 4 assembly functions just to cover the latest CPUs -- not even counting old ones. Fortunately for our sanity, these are usually templated from a single function, with small changes to the relevant areas created via macros.
Most programs aren't like codecs or even like games but could still make great gains from a properly administered dose of SIMD intrinsics.
> Inlining SIMD functions is typically not possible in most applications unless you're compiling a dozen versions, one for each possible CPU, since different CPUs will use different SIMD functions that are optimized for their performance characteristics.
I mainly care about games when it comes to high-performance programming, so this is not an issue for me. Consoles are obviously fixed platforms, and on PC any game is likely to have high enough performance requirements across the board that you can compile your executable against a SSE3 min-spec target without excluding users because of that.
> Nevermind the fact that intrinsics generally give atrocious performance, as your average 3-year-old can probably allocate registers better than gcc.
I'm glad we agree that GCC is a pile of shit (and not just in this respect). MSVC's performance with intrinsics has also gone up and down with versions but by now it's pretty solid.