Re:XLSTM: Extended Long Short-Term Memory
by brookst on Wednesday May 08, 2024 @10:01AM
(#40298228)
Congrats on the paper, very interesting.
Can you opine on how the model will fare on hardware that is optimized for transformers? There is so much investment in accelerating the transformer arch[1][2], will xLSTM / sLSTM benefit as well, or will the hardware optimizations give transformers enough of an advantage that it’s hard to compete on general purpose hardware?
Can you opine on how the model will fare on hardware that is optimized for transformers? There is so much investment in accelerating the transformer arch[1][2], will xLSTM / sLSTM benefit as well, or will the hardware optimizations give transformers enough of an advantage that it’s hard to compete on general purpose hardware?
1. https://www.etched.com/
2. https://www.embedded.com/ai-chip-features-hardware-support-f...