Re:Self-Compressing Neural Networks
by Gettingolderev on Monday August 05, 2024 @09:45AM
(#41161336)
Anyone knows if re-ordering also helps here?
If i can sort the lines of the matrix, which is probably defined by how the token embedding is setup, i could potentially zero out weights which do not contribute at all and have areas of zeros i could mark and skip?
If i can sort the lines of the matrix, which is probably defined by how the token embedding is setup, i could potentially zero out weights which do not contribute at all and have areas of zeros i could mark and skip?