Sure, if you completely throw away SIMD, things might be slow.
you are given just ~300 cycles/pixel, which is gone quickly after slow memory accesses
Memory accesses are pipelined and are generally not slow; an L1 cache access is 3 cycles and prefetching on modern CPUs is generally flawless for such situations.
multi-cycle instructions.
Since when do you need multiplies to fill a screen with a solid color? Even if you had to do YUV -> RGB conversion, that is done much faster with a multi-dimensional LUT than with multiplies (see swscale). Of course it's even faster in SIMD.
It's quite possible to do very fast rasterization on modern hardware without a GPU. It's just that people generally don't do it, because everyone has GPUs to do it for them.
Try filling any modern screen buffer (>1024x768) pixel-by-pixel, just as software rasterizers do, even with just a solid color. You'll find it takes a long and quite frankly unacceptable time (for real-time apps at least), even with modern processors.
Even at 1024x768 (@ 25 frames/sec), you are given just ~300 cycles/pixel, which is gone quickly after slow memory accesses and multi-cycle instructions.