News for hackers. Stuff that matters.

A comment on Mistral ships le chat – enterprise AI assistant that can run on prem. Parent comment

Re:Mistral ships le chat – enterprise AI assistant that can ... by aukejw on Thursday May 08, 2025 @02:41AM (#43923667)
There are plenty of smaller (quantized) models that fit well on your machine! On a M4 with 24GB it’s already possible to comfortably run 8B quantized models.

Im benchmarking runtime and memory usage for a few of them: https://aukejw.github.io/mlx_transformers_benchmark/

[ Parent | on HN ]

There are no comments on this comment yet.