News for hackers. Stuff that matters.
Re:Mistral Large 4 by throwa356262 on Tuesday October 06, 2026 @01:53PM (#49981829)
The thing with 3.8 next is that it uses a variant of ngrams. Part of the network is replaced by a lookup table you can store on a fast ssd.

In practice, you will be able to run models a bit bigger than 35B.

https://unsloth.ai/docs/models/qwen3.8-next

Re:Mistral Large 4 by cyanydeez on Tuesday October 06, 2026 @03:12PM (#49982709)
https://github.com/peonist-ai/halogen-server is beating the pants off of anything I've tried.

38GB of vram resident. more tk/s, more prefill.