News for hackers. Stuff that matters.
Re:Forget ChatGPT: why researchers now run small AIs on thei... by andai on Saturday September 21, 2024 @10:21AM (#41610167)
What local models is everyone using?

The last one I used was Llama 3.1 8B which was pretty good (I have an old laptop).

Has there been any major development since then?

Re:Forget ChatGPT: why researchers now run small AIs on thei... by esoltys on Saturday September 21, 2024 @10:34AM (#41610249)
I like [mistral-nemo](https://ollama.com/library/mistral-nemo) "A state-of-the-art 12B model with 128k context length, built by Mistral AI in collaboration with NVIDIA."
Re:Forget ChatGPT: why researchers now run small AIs on thei... by demarq on Saturday September 21, 2024 @10:37AM (#41610279)
Nada to be honest. I keep trying every new model, and invariably go back to llama 8b.

Llama8b is the new mistral.

Re:Forget ChatGPT: why researchers now run small AIs on thei... by moffkalast on Saturday September 21, 2024 @10:33AM (#41610241)
Qwen 2.5 has just released, with a surprising amount of sizes. The 14B and 32B look pretty promising for their size class but it's hard to tell yet.
Re:Forget ChatGPT: why researchers now run small AIs on thei... by alanzhuly on Saturday September 21, 2024 @05:06PM (#41612678)
I like the latest qwen2.5 (https://nexaai.com/Qwen/Qwen2.5-0.5B-Instruct/gguf-q4_0/read...). It was just released last week. It is one of the best small langauge models right now according to benchmarks. And it is small and fast!
Re:Forget ChatGPT: why researchers now run small AIs on thei... by oneshtein on Sunday September 22, 2024 @05:33AM (#41615857)
codestral 22b, aya-23 35b, gemma2 27b-instruct.