Re:Step 5 Preview, a 1M-context MoE from StepFun, shows up o...
by MisterMunchkin on Thursday October 08, 2026 @02:55PM
(#50010338)
I like trying new models but I wish we’d get something actually new. Like a new architecture or something. LLMs are just so sloppish. We can do better.
[ Parent
| on HN ]
Re:Step 5 Preview, a 1M-context MoE from StepFun, shows up o...
by famouswaffles on Thursday October 08, 2026 @04:17PM
(#50011543)
Talk is cheap. You have to actually beat transformers first. All alternate architectures that have sprung up are sidegrades at best.
Re:Step 5 Preview, a 1M-context MoE from StepFun, shows up o...
by himata4113 on Friday October 09, 2026 @08:11AM
(#50019368)
We have something better than transformers, but not better enough to shift investement to it.