Tensorflow is better than most, in that it at least implements most ops for sparse tensors, even if it doesn't have efficient implementations for things. I was recently frustrated to find that PyTorch doesn't implement a lot of basic functionality for sparse tensors at all -- it just throws an exception.
I did have great success using Netflix Vectorflow. I roughly gained 2x performance for my use case. It does leave some other performance on the table, no vectorized minibatches etc. If you are training on huge sparse datasets with a shallow ffw network, Vectorflow is a good alternative.
I do want to say a word about how hard it is to offer a set of optimized sparse representations. It is by many measures many times harder than offering an optimized dense linear algebra library. The crux is that the library or routine can not know what the actual data movement and dependencies are until the data arrives. Dense routines do not care about what the data actually is beyond its dimensions: the access pattern, compute and control flow stay exactly the same. The result is that the set of optimized sparse linear algebra routines has, on top of the usual optimization difficulties, this entire extra cartesian product of different representations and low-level operation implementation strategies. Intel MKL has a good SparseBLAS, but compare the performance of feeding it a clustered or block-sparse dataset (e.g. something out of a solver) to a large graph500 RMAT matrix.
In short, do not underestimate the great effort to offer a generally-applicable yet optimized set of sparse linear algebra routines. It makes dense algebra look really easy.
In the same way that people almost universally use BLAS and LAPACK or the MKL for linear algebra.
I realise my comment may have come off a bit flippant - it wouldn't be a trivial piece of work to write Python bindings for SuiteSparse, but would conceivably be simpler than trying to write a better sparse matrix algebra implementation.
I agree with much of your comment, but there is a great deal of serious numerics which doesn't use sparse linear algebra at all. In fact there are entire subfields of numerical analysis which involve fast algorithms for dense linear algebra.
It would be really nice if all of the sparse linear algebra in SciPy could be heavily optimized (to a similar extent as e.g. Intel optimizes their sparse operations in MKL), so that the entire ecosystem could benefit from that. This is probably something that would require some corporate support, but given how many data science and finance companies use Python for their workflow these days, it might be a wise investment.