Agreed, I've seen better results from algorithmic approaches.

For example, Takahashi for bounded-error on extremely large dot product. Depending on use case even doing a pseudo-Takahashi where you only merge the error back once every N iterations can unblock instruction-level parallelism letting the performance hit be only around 2x relative to naive dot products, but with still effectively the same guarantees.