Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model
by simonw on Thursday August 15, 2024 @06:29PM
(#41261139)
Asking "how many times does a particular thing happen in the book" is always going to be hard, because LLMs are notoriously bad at counting.
[ Parent
| on HN ]
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model
by itake on Thursday August 15, 2024 @07:18PM
(#41261525)
I had issues with gpt4 last summer confusing which characters did what and hallucinating characters.
his may of been a context/chunking issue (if that particular section doesn't name the character performing an action), but maybe its better now.
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model
by sweca on Thursday August 15, 2024 @07:26PM
(#41261587)
Were you using the API or ChatGPT? ChatGPT uses RAG on large documents which is useless for many tasks.
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model
by awwaiid on Thursday August 15, 2024 @11:33PM
(#41262920)
Last summer was like a million years ago! You should try again :)
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model
by mlyle on Thursday August 15, 2024 @07:00PM
(#41261369)
Man, -people- are bad at counting tasks like this.
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model
by campers on Thursday August 15, 2024 @11:31PM
(#41262911)
Ask it to output the occurrences as items in a JSON array, then count the array items in code
Re:Hermes 3: The First Fine-Tuned Llama 3.1 405B Model
by torginus on Friday August 16, 2024 @06:08AM
(#41264930)
Not only that, asking an LLM a question like: "List all the occurences where John Doe fights someone in the book. Also list who he fights and why" is a surefire way of getting LLMs to hallucinate, and also give incomplete info.