News for hackers. Stuff that matters.
Re:Llama 2 – Meta AI by dotancohen on Tuesday July 18, 2023 @01:52PM (#36776599)
Depends on the task. For code, ask it to implement a not-difficult but not-trivial feature. "Please add hooks to the AnkiDroid source code so that addons would be viable" might be a good start, for something that is on my mind. Then compare implementations.

For checking hallucinations, ask it about events and trivia that happened eons ago, and also within the last decade. Try some things that it can not possibly know, like how much celery Brad Pitt likes in his salad.

Re:Llama 2 – Meta AI by rajko_rad on Tuesday July 18, 2023 @02:00PM (#36776732)
This is an emerging space with lots of interesting tools coming out... There are many established benchmarks out there (i.e. included on front page of llama2 release), but most product builders have their own sets of evals that are more relevant to them...

Here is a thread exploring differences between llama-v2 vs. gpt3.5: https://twitter.com/rajko_rad/status/1681344850510376960