Most of the libs that you use are written in C with Python bindings, so they're not that slow. It's only slow if you are implementing a new algorithm without using numpy/scipy to do matrix operations.