If you train the model before you deploy it, why can’t you densely pack the nonzero weights? This is what any SparseMatrix class would do for you.