Re:Ornith-1.5: From Self-Scaffolding to Self-Improvement
by dannyw on Thursday August 20, 2026 @03:52AM
(#49371658)
Expert offloading significantly helps with the VRAM capacity.
Most MoE architectures have a few experts that are always running; this, the router, KV, and whatever else you have space for can stay in fast VRAM; and the remaining experts can be offloaded.
Most MoE architectures have a few experts that are always running; this, the router, KV, and whatever else you have space for can stay in fast VRAM; and the remaining experts can be offloaded.