Re:Task-Specific LLM Evals That Do and Don't Work
by caturopath on Wednesday December 11, 2024 @08:09PM
(#42395054)
One of the things complicating the example is that the counting-letters task is actually a legitimately hard one for it, even if it's trivial for us: it's bad at letter stuff because of the way it represents text with tokens. I believe the problem exists, but the letter thing isn't necessarily a great example of something that it should recognize as trivial.
[ Parent
| on HN ]
There are no comments on this comment yet.