The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT

Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by muyuu on Sunday April 09, 2023 @04:50PM (#35506356)

his conclusion is simultaneously not warranted and correct

a like-for-like comparison would be GPT-4 against the larger models like LLaMA 65B, but those cannot be run on consumer-grade hardware

so one ends up comparing the stuff one can run... against the top stuff from OpenAI running on high-end GPU farms, and this technology clearly benefits a lot still from much larger scale than most people can afford

the great revelation this year is how much does it get better as it get much, much bigger without a clear horizon on where will diminishing returns be hit

but at the same time, some useful stuff can be done on consumer hardware - just not the most impressive stuff

[ Parent | on HN ]

Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by Szpadel on Sunday April 09, 2023 @05:08PM (#35506484)

if "leaks" are true then even a 65B model is a toy when GPT-4 is supposed to be 1000B I would really like to have an open source model that could compete with even gpt3, but basically all lammas I tried failed to create correct email validation regex, where gpt3 is able to even refer to what rfc it is

[ Parent | on HN ]

Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by muyuu on Sunday April 09, 2023 @05:52PM (#35506821)

13B on my computer does produce useful output, it's pretty hit and miss though

GPT-3.5 OTOH is much better, but it's also much better at producing convincing-sounding but completely incorrect answers

[ Parent | on HN ]

Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by wokwokwok on Monday April 10, 2023 @01:00AM (#35509326)

The 65B model runs fine on a Mac Studio with 64GB of memory.

The output is unremarkable; it’s not significantly better than the 13B model for most uses.

GPT 3.5 is an order of magnitude better at least.

[ Parent | on HN ]

Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by muyuu on Monday April 10, 2023 @09:42AM (#35512171)

You cannot heavily quantise models and assume they didn't degrade significantly.

To run it properly you need a lot more than a Mac Studio, and then comparisons need to be done more or less seriously, not just a few random prompts, because anything in a black box will "cheat" and will be fine tuned to do well at popular benchmarks.

[ Parent | on HN ]