Re:70B Llama 2 at 35tokens/second on 4090
by andai on Wednesday September 13, 2023 @03:50AM
(#37493512)
It seems to depend on the task. I'd say it beats GPT-3 in quality most of the time (but not in speed and cost!) and GPT-4 approximately never. It's perfect if what you need is "less good than GPT-4 but 50x cheaper."
[ Parent
| on HN ]
Re:70B Llama 2 at 35tokens/second on 4090
by wkat4242 on Wednesday September 13, 2023 @07:53AM
(#37495509)
The biggest benefit of llama is that you can get it uncensored so you don't get the weasely lawyer disclaimer crap all the time.
Re:70B Llama 2 at 35tokens/second on 4090
by andai on Thursday September 14, 2023 @12:27PM
(#37511074)
In my experience the Chat version of Llama 2 is significantly worse than the GPTs, on the "I'm afraid I can't do that" front. It doesn't lecture you as much, but suggests "how about we do [something unrelated and useless] instead?"
As for the plain text prediction version (which is the only uncensored one?), I haven't been able to get it to do anything useful, even when I provide examples (seems dumber than even ancient GPT-3?).
Also, I got some bizarre and disturbing outputs from the uncensored version, like it was trained on some very nasty inputs! I assume that's why they went so hard on the safety phase to compensate...