Mistrals main strength is that it is a good allrounder model with a 7B size, so it runs on a lot of consumer hardware. If you are looking for specific tasks to run locally (with decent RAM and no GPU), the realistic best option in most categories is a fine-tuned 13B model.

In my experience most open source models do quite well with Python and Javascript, but don't perform so well when it comes to other languages and the usage of their special characteristics (e.g. it will still be able to write simple control flow in Rust, but probably not use async/await correctly).

I've tried using LLMs in general with IaC (K8s resources/Helm charts), and they all did well when asking it a very specific thing about it (e.g. "Is there an alternative way to accomplish X?"), but also never had it perform well when outputting YAML directly (either from scratch of modifying it).

I never heard of IaaC before.

I assume you mean IaC? It’s how you manage infrastructure with automation. Terraform, Ansible, CDK etc… back in the day it used to be for managing pets with Pupper, Chef and friends.

Excellent at creative writing

Pretty good at instructions

Uncensored, no RHLF bs

Tell me how can I run it locally.

Ashu, i am going to guess that english is not your first language, apologies, if that is not the case, but " Tell me how can I run it locally." is not a great way to ask for info, not here at least. A 'please' would certainly help, but a less direct way might be even better.

Anyway, to answer your question: https://ollama.ai/

Which did you try? Many of these models don't work well at all for queries like "please do this task for me". But then people fine tune them and they work much better.

Give zephyr a try (available in ollama and similar places)

It's a fine tune of mistral and works quite well.

But as you point out, these models have less general knowledge compared to their massive siblings. Knowledge based queries are going to be lower quality than using gpt-4.

I tried the basic Mistral Instruct model. I'll give Zephyr a try, thanks for the recommendation!