Re:Step 5 Preview, a 1M-context MoE from StepFun, shows up o...
by Iolaum on Friday October 09, 2026 @09:26AM
(#50020225)
Indeed - Even with llama.cpp you can store n-grams to disk and use it. I m using unsloth's llama fork atm but a new llama release is out that may be better - hadn't had time to test it yet.
It is fully possible to run the 180B Owen model on strix halo using strata and even get reasonable speed out of it.
https://github.com/Niko1221/Strata