News for hackers. Stuff that matters.
Re:Step 5 Preview, a 1M-context MoE from StepFun, shows up o... by MisterMunchkin on Thursday October 08, 2026 @02:55PM (#50010338)
I like trying new models but I wish we’d get something actually new. Like a new architecture or something. LLMs are just so sloppish. We can do better.
Re:Step 5 Preview, a 1M-context MoE from StepFun, shows up o... by famouswaffles on Thursday October 08, 2026 @04:17PM (#50011543)
Talk is cheap. You have to actually beat transformers first. All alternate architectures that have sprung up are sidegrades at best.
Re:Step 5 Preview, a 1M-context MoE from StepFun, shows up o... by himata4113 on Friday October 09, 2026 @08:11AM (#50019368)
We have something better than transformers, but not better enough to shift investement to it.
Re:Step 5 Preview, a 1M-context MoE from StepFun, shows up o... by cyanydeez on Thursday October 08, 2026 @03:10PM (#50010551)
welcome to the sigmoid...