News for hackers. Stuff that matters.
Re:Llamafile lets you distribute and run LLMs with a single ... by int_19h on Thursday November 30, 2023 @05:08AM (#38471731)
The best models available to the public are only slightly better than the original (pre-turbo) GPT-3.5 on actual tasks. There's nothing even remotely close to GPT-4.
Re:Llamafile lets you distribute and run LLMs with a single ... by pyinstallwoes on Thursday November 30, 2023 @06:09AM (#38472095)
What’s the best in terms of coding assistance? What’s annoying about gpt 4 is that is seems badly nerfed in many ways. It is obviously being conditioned in its own political bias.
Re:Llamafile lets you distribute and run LLMs with a single ... by dmd on Thursday November 30, 2023 @08:51AM (#38473500)
... is the javascript it's writing for you too 'woke' or something?
Re:Llamafile lets you distribute and run LLMs with a single ... by pyinstallwoes on Thursday November 30, 2023 @11:32AM (#38475509)
Lately it’s been refusing to write code at all. // implementation details here.

I think it’s the opposite of woke, it’s slept!

Re:Llamafile lets you distribute and run LLMs with a single ... by eyegor on Thursday November 30, 2023 @08:55AM (#38473536)
In my experience, the deepseek-coder-instruct family is at least as good as gpt-3.5-turbo on python. Even the 1.3b models are very good (and run okay on cpu), although you should use larger if you have the vram. There are even larger models (30b+) if you are drowning in vram, but I don't think they perform much better at coding than deepseek-coder-instruct 6.7b.

3-4gb vram or cpu (1.3b): https://huggingface.co/TheBloke/deepseek-coder-1.3b-instruct...

Alternative for chat (1.3b): https://huggingface.co/TheBloke/evolvedSeeker_1_3-GGUF

Alternative for chat (3b): https://huggingface.co/TheBloke/open-llama-3b-v2-wizard-evol...

6-8gb vram (6.7b): https://huggingface.co/TheBloke/deepseek-coder-6.7B-instruct...

Re:Llamafile lets you distribute and run LLMs with a single ... by rightbyte on Thursday November 30, 2023 @05:46PM (#38480309)
Really? How could this model not have had front page articles on HN? Is the self-hosted one as good as the one or their website?
Re:Llamafile lets you distribute and run LLMs with a single ... by eyegor on Friday December 01, 2023 @09:16PM (#38495214)
Hmm, I just assumed people knew about it. Submitted https://news.ycombinator.com/item?id=38495176

Self hosted 6.7b model is phenomenal in my experience, I actually prefer it to chatgpt a lot of the time. Similar quality code but fewer disclaimers and #todo in my experience. Gpt4 is still a bit better at coding but not by much and it's much slower for me.

Re:Llamafile lets you distribute and run LLMs with a single ... by int_19h on Thursday November 30, 2023 @02:37PM (#38478070)
The best in terms of coding assistance, and really for anything else, is the original (pre-turbo) GPT-4 used via the API, although this will also be more costly. There are many third-party chat apps that are wrappers around that now if you want a ChatGPT-like experience.

This can also significantly reduce its bias since you are in control of the system prompt. But also, even ChatGPT can be trivially made to behave differently by saying that you're writing a book or making a video game etc, describing a character in it, and then asking it how that character would have responded in such and such situation.