News for hackers. Stuff that matters.

A comment on Ask HN: Is anybody building an alternative transformer?. Parent comment

Re:Ask HN: Is anybody building an alternative transformer? by bob1029 on Friday February 14, 2025 @03:30PM (#43052766)
I was just about to post this. There was a MLST podcast about it a few days ago:

https://www.youtube.com/watch?v=8u2pW2zZLCs

Lots of related papers referenced in the description.

[ Parent | on HN ]
Re:Ask HN: Is anybody building an alternative transformer? by RossBencina on Friday February 14, 2025 @09:46PM (#43055416)
One claim from that podcast was that the xLSTM attention mechanism is (in practical implementation) more efficient than (transformer) flash attention, and therefore promises to significantly reduces the time/cost of test-time compute.
[ Parent | on HN ]
Re:Ask HN: Is anybody building an alternative transformer? by korbip on Friday February 14, 2025 @10:14PM (#43055573)
Test it out here:

https://github.com/NX-AI/mlstm_kernels

https://huggingface.co/NX-AI/xLSTM-7b

[ Parent | on HN ]