News for hackers. Stuff that matters.
Re:Leave No Context Behind: Efficient Infinite Context Trans... by Manabu-eo on Thursday April 11, 2024 @05:57PM (#40007257)
How it compares to the many hybrid SSM/Transformer architectures that are coming out now, with 2B+ weights released? I skimmed the paper, but it seems it's innovation is basically combining both in the same layer, instead of alternating layers, but I see no reference to the latter.

There are no comments on this comment yet.