News for hackers. Stuff that matters.
Re:Elon Musk wanted an OpenAI for-profit by Flomolok on Friday December 13, 2024 @03:46PM (#42412195)
I tried Claude.

If hardware continues it's evolution of speed in the next 10 years I can have Claude but local + running constantly and yeah that would change certain things fundamentaly

Re:Elon Musk wanted an OpenAI for-profit by jasonjmcghee on Friday December 13, 2024 @04:02PM (#42412322)
Try llama 3.3 70B. On groq or something. Runs on a 64GB macbook (4bit quantized, which seems to not impact quality much). Things have come a long way. Compare to llama 2 70b. It's wild
Re:Elon Musk wanted an OpenAI for-profit by Terretta on Friday December 13, 2024 @04:55PM (#42412714)
Llama 3.3 70B 8-bit MLX runs on Macbook 128GB at 7+ tokens per second while running a full suite of other tools, even at the 130k tokens size, and behaves with surprising coherence. Reminded me of this time last year, first trying Mixtral 8x22 — which still offers a distinctive je ne sais quoi!
Re:Elon Musk wanted an OpenAI for-profit by Scene_Cast2 on Friday December 13, 2024 @03:51PM (#42412229)
When ChatGPT was down a few days back, I locally booted up Codestral. It was decent and usable.
Re:Elon Musk wanted an OpenAI for-profit by nmfisher on Friday December 13, 2024 @11:17PM (#42414646)
Qwen 2.5 32B Coder is actually a viable locally-hosted alternative to Claude 3.5 Sonnet.

It's not better, but if I couldn't access Claude for some reason, I would definitely use it.