Is anyone able to find evaluation of these models besides speed and power consumption? I really cant find any baseline comparisons of accuracy and so on for any relevant datasets being used these days. I wish they did a better job of presenting that since right now I'm thinking I could build a million times faster NN with 0% accuracy too...

If you train the model before you deploy it, why can’t you densely pack the nonzero weights? This is what any SparseMatrix class would do for you.