C, however, gives you hold onto the generator to say "use this assembly". So, for new or one off instructions, it is a quick way to get an edge.
When you use inline assembly, the compiler (more or less) has to execute that inline assembly, right where you say it is. It can't speculatively execute it, it can't refactor the call into a different function if you're calling it twice, it can't optimize it out in certain circumstances, etc. It can't move the instruction around or choose a different form depending on register pressure. If you specify the right flags about what registers it clobbers, what memory barriers it requires, etc., the compiler can do some of it, but it's not going to be nearly as good as the compiler actually understanding what's happening, I think.
Telling the code generator (and C is a high-level language in this respect, if you want any performance out of C) what you're doing is likely to get you more of an edge, no?
I can certainly imagine cases where letting the compiler reorder a slow square root around a memory access will be faster in practice than a fast square root that can't be reordered.
It is not exclusive to C compilers.
Isn't the right way to handle this for your code generator, e.g., LLVM, to know about this instruction, and your language implementation, e.g., clang or rustc, to just say "hey, give me a square root"?
I do buy your argument for very processor-specific things like writing a bootloader or a hypervisor.