News for hackers. Stuff that matters.
Re:Task-Specific LLM Evals That Do and Don't Work by Havoc on Monday December 09, 2024 @10:22AM (#42367018)
A lot of models have also been overly chat trained. Responding with stuff like “sure I can help you with that”

That’s just unwanted noise if you’re trying to use them as a code building block in an application. So you need to force json or similar…which I suspect harms accuracy over free form

Re:Task-Specific LLM Evals That Do and Don't Work by TeMPOraL on Monday December 09, 2024 @11:32AM (#42367687)
Unfortunately, that "unwanted noise" is a space for the models to compute; trying to eliminate it gives suboptimal responses. What you can do instead is try to corral it - let the model "think" like it wants, but guide it to add markers wrapping the thinking and/or result, then filter out the thinking in UI (for interactive applications) or as an intermediate/post-processing step (for hidden "building blocks").

If you're using Anthropic models, you may actually get improvements from prompting the model to maintain a tagging discipline; see https://docs.anthropic.com/en/docs/build-with-claude/prompt-....

Re:Task-Specific LLM Evals That Do and Don't Work by iknownthing on Monday December 09, 2024 @11:34AM (#42367704)
interesting
Re:Task-Specific LLM Evals That Do and Don't Work by hedgehog on Monday December 09, 2024 @12:39PM (#42368303)
As other people pointed out here you can also add "verbosity sinks" as text fields in structured output, recently I've also been experimenting with tool calls to support guided self-talk in a way that doesn't necessarily all accumulate in the context (e.g. if not all the tool parameters get echoed back).
Re:Task-Specific LLM Evals That Do and Don't Work by glaugh on Monday December 09, 2024 @01:50PM (#42369181)
Thank you (and teMPOral) for these comments, this sounds potentially useful to me.

I hate to ask this, but I'm struggling to find any thorough posts or articles or papers about this, do you have any links you could point me toward?

Re:Task-Specific LLM Evals That Do and Don't Work by hedgehog on Monday December 09, 2024 @04:37PM (#42370786)
Speaking only for myself these ideas are a combination of things I've seen scanning new papers and informal discussions with other people working in the area. Feel free to shoot me an e-mail though, maybe I can point you somewhere more specific.

Edit: The "verbosity sink" name is inspired by the idea from the paper below although they're not actually at all the same thing.

https://arxiv.org/abs/2309.17453

Re:Task-Specific LLM Evals That Do and Don't Work by jacobr1 on Monday December 09, 2024 @07:34PM (#42372283)
Here is a short example that came up for me last week.

I had a set of documents I wanted to classify according a taxonomy that is well known (so it is exists in the training data of all the major llm models I tested)

If I have prompt like, `You are an expert classification system. Using the Classification Approach Foo, consider the following and output the category in JSON format, such as {"class":"bar"} `

This works ok, but it works much better if I tell it to output {"class":"bar", "reason": "baz"} and improved with some other approaches like adding "related_class" or "parent_category" which would otherwise be redundant.

Also including some few-shot examples helped, but the biggest benefit came from the "reason" field. Trying justification or other synonyms seems to produce the same output.

I suspect this is something similar to CoT.

Re:Task-Specific LLM Evals That Do and Don't Work by glaugh on Monday December 09, 2024 @09:28PM (#42373085)
Great, thank you (and hedgehog), that makes sense
Re:Task-Specific LLM Evals That Do and Don't Work by potatoman22 on Monday December 09, 2024 @11:33PM (#42373788)
Have you tested moving the "reason" field before the "class" field? That may encourage better CoT instead of having the model justify the class after it already picked it. Anecdotally, I saw a 5% boost in performance from a NER system from having the model output the entity's class at the end rather than the beginning.
Re:Task-Specific LLM Evals That Do and Don't Work by te_chris on Tuesday December 10, 2024 @04:30AM (#42375235)
This has worked for me for sentiment classification, i had a score and a summary field and the results were much more accurate.
Re:Task-Specific LLM Evals That Do and Don't Work by andai on Tuesday December 10, 2024 @10:05AM (#42377521)
Does it still work if the class field is output before the reason?
Re:Task-Specific LLM Evals That Do and Don't Work by jacobr1 on Tuesday December 10, 2024 @12:06PM (#42378732)
Yes. But, reason first did work (very slightly) better.
Re:Task-Specific LLM Evals That Do and Don't Work by imtringued on Monday December 09, 2024 @01:26PM (#42368902)
We have Marco o1 at home.
Re:Task-Specific LLM Evals That Do and Don't Work by behnamoh on Monday December 09, 2024 @02:25PM (#42369542)
Re:Task-Specific LLM Evals That Do and Don't Work by TeMPOraL on Monday December 09, 2024 @03:41PM (#42370350)
That's... a good result, actually. No, I'm serious.

This reads exactly like my inner thought process on a novel or tricky task I'm asked to solve, especially when I know I'm tired (or drunk, back in the times I consumed alcohol on a regular basis), and need to spell everything out (out loud or in a text file).

Hell, it's exactly how I expect a kid who just learned about fractions would think. I have a vague recollection I processed such tasks this explicitly as a kid, until I understood the topic.

LLMs pulling this off reliably? That's huge progress. I used to say[0] that GPT-4 is best imagined as a 4 year old kid that memorized half the Internet. But this? This is 8 year old's stuff.

--

[0] - I currently prefer comparing it to "inner voice", and its performance and propensity to hallucinations to a smart schoolkid that's being asked questions by the teacher about things they only read about but didn't fully process, and who's pressured into giving some answer, as saying "I don't know" is an instant F and public humiliation. Such kid will be forced to extrapolate on the spot, but if they're smart enough and remember enough, they'll often get it at least partially right. I know that from personal experience :).

Re:Task-Specific LLM Evals That Do and Don't Work by caturopath on Tuesday December 10, 2024 @06:39AM (#42375936)
Yeah, I don't know if someone thought this was bad or something, but it seems like valid reasoning. We may need to give these models a sense of things that require more and less detailed reasoning and better knowledge of what they know internally, but the "research work" the model claims to be, it seems like it's doing a good job.

The poster also shared in a comment https://preview.redd.it/u8vs29hq5w2e1.png?width=2704&format=... which did get the intended laugh out of me, but even that seems fair enough. I'm currently traveling in a country where most people speak a language I don't know well. You better believe I've been thinking through even trivial greetings, considering the setting, formality, appropriate follow ups, etc.

Re:Task-Specific LLM Evals That Do and Don't Work by behnamoh on Tuesday December 10, 2024 @10:22AM (#42377660)
OP here. I liked Macro-o1 (Marco1?) but as you pointed out, we need to teach these models to spend their system 2 thinking more economically.
Re:Task-Specific LLM Evals That Do and Don't Work by caturopath on Wednesday December 11, 2024 @08:09PM (#42395054)
One of the things complicating the example is that the counting-letters task is actually a legitimately hard one for it, even if it's trivial for us: it's bad at letter stuff because of the way it represents text with tokens. I believe the problem exists, but the letter thing isn't necessarily a great example of something that it should recognize as trivial.
Re:Task-Specific LLM Evals That Do and Don't Work by TeMPOraL on Tuesday December 10, 2024 @01:09PM (#42379457)
Looking at that example, I too feel it's a thought process I could go through once or twice. I think this highlights an important difference between humans and LLMs: a human can think such things explicitly in their head, or on paper (I do it in text editor more frequently than I'd care to admit), once or twice, and then it sticks, quickly becomes more of a "system 1" thing. With LLMs, the closest to that outside training/fine-tuning would probably be prompt caching. It would be great if we could figure out some kind of on-line learning scheme so the model could internalize its own thoughts and persist them between conversations, but in the latent space, not prepended as token input.
Re:Task-Specific LLM Evals That Do and Don't Work by caturopath on Wednesday December 11, 2024 @08:10PM (#42395061)
> ou better believe I've been thinking through even trivial greetings

Even after thinking through what to say, I used the wrong greeting in a shop half an hour ago and the person working there called me on it.

Re:Task-Specific LLM Evals That Do and Don't Work by pizza on Monday December 09, 2024 @01:26PM (#42368912)
It seems to me that it would make sense to just include more <BOS>-like meta tokens at the beginning in such cases, and have them as a prefixed scratch space that can be suppressed by treating them as non-output tokens.
Re:Task-Specific LLM Evals That Do and Don't Work by bubaumba on Monday December 09, 2024 @08:45PM (#42372786)
it should be possible to ask model to think aloud (or step-by-step) and then give summary. in one or two prompts. give only summary back to user.
Re:Task-Specific LLM Evals That Do and Don't Work by Jerrrry on Friday December 13, 2024 @11:19AM (#42409886)
It may be the self-aware human bias tainting this reasoning, but it seems convergent with our own psyche/brain processes, and/or inherent to the way we commonly express conscious thoughts.

Percolating tokens that allow a more "accurate" latent space appear to be more accurate, but are nearly actually useless noise. Almost a virtual shower thought.

Because people only put the answer at the end of a grammatically correct statement, with the more "reasoned" statements being more articulately percolated/logically sound, and that is expressed grammatically. These statements are inferred to be associated with intellectual boiler-plate. They may be correlated and not actually causative, but that would require a multiple component architecture with embeddings being used as a proto-"qualia" and that is getting hairy.

Facts should "only" have to be read once, and should be explicitly defined with a more secure of a confidence completely. Implicit inferences from those explicit facts should be emitted from a different, less confident module; with the chat boilerplate being tacitly composed finally when presenting the output to the user.

Of course separating the baby from the bathwater is the hard (not impossible) part.

Re:Task-Specific LLM Evals That Do and Don't Work by msp26 on Monday December 09, 2024 @10:34AM (#42367141)
> which I suspect harms accuracy over free form

Untrue in my testing. If you want to use chain of thought, you can always throw in a `thoughts` field (json field/xml tags) before the rest of your output.

Re:Task-Specific LLM Evals That Do and Don't Work by n2d4 on Monday December 09, 2024 @10:46AM (#42367273)
If you want to be really sure, you can also first ask it to respond in chat format, and then ask it again to respond in JSON format, if you can afford the cost.
Re:Task-Specific LLM Evals That Do and Don't Work by msp26 on Monday December 09, 2024 @10:57AM (#42367376)
It really isn't necessary when using constrained decoding (aka structured outputs) which guarantees that you'll get JSON output in the correct structure.
Re:Task-Specific LLM Evals That Do and Don't Work by qeternity on Monday December 09, 2024 @12:14PM (#42368045)
This is not true at all. Just because you can force the logits to give syntactically valid outputs, doesn't mean you're going to get a useful result.

Constrained generation, without a proper understanding of the model's natural response tendencies, can give horrible results.

Re:Task-Specific LLM Evals That Do and Don't Work by msp26 on Monday December 09, 2024 @01:07PM (#42368684)
I agree with you completely. I was talking about the parsing being easy with this, not referring to the outputs being correct in reality.

You can get awful results with poorly defined constraints.

Re:Task-Specific LLM Evals That Do and Don't Work by imtringued on Monday December 09, 2024 @01:40PM (#42369055)
Depends on the way you do constrained generation. If all you do is reject tokens using a grammar, then yeah it is bad. If your software inserts things like field names and braces instead of forcing the model to produce them token by token and then afterwards rejecting the wrong tokens, then you should be good to go.
Re:Task-Specific LLM Evals That Do and Don't Work by potatoman22 on Monday December 09, 2024 @11:34PM (#42373794)
Why is it bad to reject tokens using a grammar?
Re:Task-Specific LLM Evals That Do and Don't Work by phillipcarter on Monday December 09, 2024 @10:31AM (#42367106)
I've not had that experience when I include in the prompt for a coding LLM "only respond with the code".

Though it's worth noting that I often do want an explanation, and currently my workflow is to just use a different LLM.

Re:Task-Specific LLM Evals That Do and Don't Work by michaelt on Monday December 09, 2024 @12:22PM (#42368112)
There were some models in the past [1] that were extremely keen to produce chatty noise, even when you explicitly asked them not to.

Of course this was back in May 2023, so things might have improved since then.

[1] https://news.ycombinator.com/item?id=35964018

Re:Task-Specific LLM Evals That Do and Don't Work by Kuinox on Monday December 09, 2024 @11:09AM (#42367492)
Are you not using instruct tuned models ?
Re:Task-Specific LLM Evals That Do and Don't Work by TeMPOraL on Monday December 09, 2024 @11:27AM (#42367641)
Obviously they are, that's why they have this problem. Or did the terms "instruction tuning" and "instruct models" change their meaning when I wasn't looking?
Re:Task-Specific LLM Evals That Do and Don't Work by knicholes on Monday December 09, 2024 @11:41AM (#42367754)
Shoot, maybe someone edited something, but I don't see anyone else in this conversation using the terms "instruction tuning" and "instruct models"?
Re:Task-Specific LLM Evals That Do and Don't Work by petesergeant on Monday December 09, 2024 @11:11AM (#42367505)
This isn’t a problem in practice. Most of my prompts ask the LLM to do a bunch of chain of thought before asking them to spit out JSON. I extract the JSON, which works 97.5% of the time, and have a retry step being real specific about “here’s the conversation so far but I need JSON now” that handles the rest. Adding examples really helps.
Re:Task-Specific LLM Evals That Do and Don't Work by imtringued on Monday December 09, 2024 @01:43PM (#42369098)
https://lmsys.org/blog/2024-02-05-compressed-fsm/

I'm not trying to shill sglang specifically, just pointing out that there's a better way, btw.

Re:Task-Specific LLM Evals That Do and Don't Work by hansvm on Monday December 09, 2024 @02:39PM (#42369686)
...with the obvious caveat that the distribution of responses isn't the same

Elaborating slightly, retrying till the schema is adhered to has a different distribution from greedily selecting tokens adhering to the schema.

The simplest toy example I can come up with for that property is a universe of answers "aa", "ab", "bc", all of which the model is equally likely to output for a given prompt with normal auto-regressive invocations. The schema, in regex, is ".[bc]". Retry-till-success produces "ab" 1/2 of the time and "bc" the other half. Greedily adhering to the schema produces "ab" 2/3 of the time and "bc" the remaining third.

Last I checked large-scale LLMs, it was a problem in the wild for large string fields. They tend to want to finish the string with ellipses (this creating an incorrect response), but when they made that mistake they'd tend to truncate the entire json record and generate something that doesn't adhere to the schema. Retry-till-success has a high successful parse rate. Greedily adhering to the schema converts those ellipses errors into syntactically correct garbage.

Other such bugs can be much harder to quantify (model explainability is hard), but I'd be cautious employing the technique without a lot of case studies for your particular problem domain.

Re:Task-Specific LLM Evals That Do and Don't Work by ActionHank on Monday December 09, 2024 @02:25PM (#42369538)
I also firmly believe that number of tokens served is a metric that is tracked and encouraged to go up, because more tokens mean more charges. o1 "does more" by using a whole lot more tokens for a very slight bump in usefulness.
Re:Task-Specific LLM Evals That Do and Don't Work by baq on Tuesday December 10, 2024 @01:35AM (#42374369)
The problem with such takes is 1) the facts are also consistent with performance plateauing for fundamental reasons and 2) there are good, perhaps better than them, competitors with similar issues.
Re:Task-Specific LLM Evals That Do and Don't Work by ActionHank on Tuesday December 10, 2024 @05:41AM (#42375605)
I think we’re on the same page here. In lieu of growing the product because of fundamental constraints they’ve opted to grow business metrics to net more revenue which in turn looks good on paper to investors.