Every story and comment from the Hacker News API, in a layout borrowed from 2001.
A comment on Mlx-community/OLMo-2-0325-32B-Instruct-4bit.
How do mlx quants compare to gguf?
Edit: the thread below says that mlx is faster, but gguf quantisations tend to maintain better text quality.
https://www.reddit.com/r/LocalLLaMA/comments/1gc0t0c/how_doe...
There are no comments on this comment yet.
How do mlx quants compare to gguf?
Edit: the thread below says that mlx is faster, but gguf quantisations tend to maintain better text quality.
https://www.reddit.com/r/LocalLLaMA/comments/1gc0t0c/how_doe...