It won't really help. You'll get the same issues with stiff DEs or ill-conditioned matrices for perfect precision or floating-point. There will be some cases where it will help, but not a lot.

I've implemented a few DE solvers and sparse matrix implementations before. I think sparse matrices is one case where it would help.

Yes, this won't change the challenging nature of solving the underlying problem.

More problematic is that when a matrix is ill-conditioned it may not be all that helpful to have "an answer" since the ill-conditioning tells you there are many solutions closely satisfying the system of equations.