Mistral ships le chat – enterprise AI assistant that can run on prem

Re:Mistral ships le chat – enterprise AI assistant that can ... by jychang on Wednesday May 07, 2025 @09:41PM (#43922306)

16GB on a mac with unified memory is too small for good coding models. Anything on that machine is severely compromised. Maybe in ~1 year we will see better models that fit in ~8gb vram, but not yet.

Right now, for a coding LLM on a Mac, the standard is Qwen 3 32b, which runs great on any M1 mac with 32gb memory or better. Qwen 3 235b is better, but fewer people have 128gb memory.

Anything smaller than 32b, you start seeing a big drop off in quality. Qwen 3 14b Q4_K_M is probably your best option at 16gb memory, but it's significantly worse in quality than 32b.

[ Parent | on HN ]

Re:Mistral ships le chat – enterprise AI assistant that can ... by chedabob on Thursday May 08, 2025 @05:24AM (#43924502)

What do you use to interface with Qwen?

I have LMStudio installed, and use Continue in VSCode, but it doesn't feel nearly as feature rich compared to using something like Cursor's IDE, or the GitHub Copilot plugin.

[ Parent | on HN ]

Re:Mistral ships le chat – enterprise AI assistant that can ... by Lalabadie on Thursday May 08, 2025 @08:22AM (#43925430)

Continue can be your autocomplete provider – and use a smaller and faster model. Something like Cline (or Roo or Kilocode or another fork) would be the more Cursor-like assistant there.

[ Parent | on HN ]