News for hackers. Stuff that matters.

A comment on The LLama Effect: Leak Sparked a Series of Open Source Alternatives to ChatGPT. Parent comment

Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by wokwokwok on Monday April 10, 2023 @01:00AM (#35509326)
The 65B model runs fine on a Mac Studio with 64GB of memory.

The output is unremarkable; it’s not significantly better than the 13B model for most uses.

GPT 3.5 is an order of magnitude better at least.

[ Parent | on HN ]
Re:The LLama Effect: Leak Sparked a Series of Open Source Al... by muyuu on Monday April 10, 2023 @09:42AM (#35512171)
You cannot heavily quantise models and assume they didn't degrade significantly.

To run it properly you need a lot more than a Mac Studio, and then comparisons need to be done more or less seriously, not just a few random prompts, because anything in a black box will "cheat" and will be fine tuned to do well at popular benchmarks.

[ Parent | on HN ]