As reported by the article: "However, for smaller matrices, we’re doing poorly in comparison to Nvidia’s library! This happens because cuBLAS contains not one single implementation of SGEMM, but hundreds of them.", it would take considerable effort to replace CuBLAS.