A while ago, I was doing some exploratory work — basically trying out various approaches to a problem. There was a particular piece of code that was quite expensive to execute and that was executed a lot. The net result was that a single calculation of the required matrix took about two months.

It took more than a month of hard, full-time work to fit the problem into fixed-point registers and the SIMD model used by Intel SSE. Over two weeks from that month were spent on debugging, even though I went through multiple versions in C, each one closer to the target assembly. But the resulting speedup was well worth it: about 600x.

Now think about it. 600x. We went from "a single try/iteration takes two months" to "we can try six variations overnight".

So yes, assembly matters, for many reasons of which I only outlined one.

As a side note, people tend to focus on big-O algorithm performance, neglecting the constant. But the constant matters in practice, more than most people think.