Compilers are very good in some things like instruction scheduling and register allocation but only poor to mediocre in other things like instruction selection. In addition, compilers are not allowed to do some optimizations, like change the order of function calls to other translation units or do arithmetic optimizations with floating point numbers.
So if you have a piece of performance critical code, you can still be smarter than the compiler alone by co-operating with your compiler by writing compiler-friendly close to the metal code. Inline assembly code is not good for the compiler, it can't really optimize it at all. But using CPU and compiler specific intrinsics to use your hardware capabilities (SIMD, special cpu instructions) you can get your code to run close to the speed of light without compromising readability or maintainability.
Yes, compilers these days are smart but thinking that you can't do better than that is a lazy man's fallacy.
Actually it can, if you have a way to specify register usage (gcc does) and the programmer gives the right specification. Unfortunately, that almost never seems to happen in practice.
Yeah, I found this out the hard way. Turns out it's really ridiculously difficult to get GCC to put the rdtsc instruction where you want it :)
But you get the point, the compiler can optimize better if you use C and intrinsics than inline asm.
We had a former performance guru who once put a huge #ifdef I_DONT_CARE_ABOUT_PERFORMANCE around the "naive" C version of a loop and had his favourite x86 asm version on the #else branch of the ifdef. Needless to say, much hilarity resulted when defining this macro improved performance. I suspect that the naive version may have been a bad idea at one stage (Pentium 4, perhaps) but became a better version from Core 2 onwards...
Ocassionally I'll get on a wild tear and think "Oh, I can do better than the compiler" after it generates some ridiculous-looking instruction sequence that seems to not be using enough registers (or whatever). I then find that hand-writing and hand-scheduling everything in accordance to what looks really good to me - actually works worse than the insane-looking code in the compiler. Put it down to stuff like store-to-load forwarding, dynamic scheduling, register renaming, etc.