News for hackers. Stuff that matters.
Re:GPT-4 API General Availability by PostOnce on Thursday July 06, 2023 @10:43PM (#36626409)
I'm using a variety of 7 and 13B models (and a 3B one for fast feedback loop debugging) at between 8bit and 4_K_M quantizations.

Depending on your pre-prompt, your fine-tune (i.e. which model you downloaded), and your specific task, the results can be startlingly good, it's crazy that you can do this on a $250 laptop. I stay up nights working on it lately, it's so interesting.

More importantly, things change by the day. New models, new methods, new software, new interfaces... the possibilities are endless... unless we let OpenAI corrupt our government(s).

Re:GPT-4 API General Availability by redox99 on Friday July 07, 2023 @03:50AM (#36628560)
I'm surprised you're having such a good time with 7B and 13B models. I find anything below 33B to be almost useless. And only 65B is close to GPT 3.5.
Re:GPT-4 API General Availability by mdale on Friday July 07, 2023 @09:45AM (#36631673)
I don't think the "corrupt our government" thing is going to happen . The wave of change is too large the tech is moving too fast and into evey facet of data and software. There is competition globally and locally; a regulatory slow down is unlikely.