News for hackers. Stuff that matters.
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model by simonw on Thursday August 15, 2024 @06:29PM (#41261139)
Asking "how many times does a particular thing happen in the book" is always going to be hard, because LLMs are notoriously bad at counting.
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model by itake on Thursday August 15, 2024 @07:18PM (#41261525)
I had issues with gpt4 last summer confusing which characters did what and hallucinating characters.

his may of been a context/chunking issue (if that particular section doesn't name the character performing an action), but maybe its better now.

Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model by sweca on Thursday August 15, 2024 @07:26PM (#41261587)
Were you using the API or ChatGPT? ChatGPT uses RAG on large documents which is useless for many tasks.
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model by awwaiid on Thursday August 15, 2024 @11:33PM (#41262920)
Last summer was like a million years ago! You should try again :)
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model by mlyle on Thursday August 15, 2024 @07:00PM (#41261369)
Man, -people- are bad at counting tasks like this.
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model by campers on Thursday August 15, 2024 @11:31PM (#41262911)
Ask it to output the occurrences as items in a JSON array, then count the array items in code
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model by torginus on Friday August 16, 2024 @06:08AM (#41264930)
Not only that, asking an LLM a question like: "List all the occurences where John Doe fights someone in the book. Also list who he fights and why" is a surefire way of getting LLMs to hallucinate, and also give incomplete info.