Re:Jsonformer: A bulletproof way to generate structured outp...
by ianbutler on Tuesday May 02, 2023 @05:39PM
(#35794591)
* * *
Re:Jsonformer: A bulletproof way to generate structured outp...
by killthebuddha on Tuesday May 02, 2023 @09:41PM
(#35796660)
I think I agree, and am doing something similar as well. Even the adapters approach does require some amount of consistency across prompts. For example, if you have one prompt that says “you are a very helpful assistant” and one prompt that says “you are a very lazy assistant”, then even if those prompts are otherwise written to be as orthogonal as possible you will still probably see degradation in completion quality.
Re:Jsonformer: A bulletproof way to generate structured outp...
by newhouseb on Tuesday May 02, 2023 @10:20PM
(#35796954)
I haven't spent time going deep here but my current hypothesis is that interoperability will more or less end up looking like toolformer where the "tools" are just separate LLM runs with task-specific context. So for example:
> Instruction: Write a poem and them emit a structure that follows a schema named X.
> Completion: [map-schema X "roses are red, violets are blue"]
Conceptually this is basically just a function call where context is local to the function.
MEMORY EXAMPLE INSTRUCTION [COMPLETION]
it will basically not work to wrap it in a prompt that's structured
INSTRUCTION MEMORY EXAMPLE [COMPLETION]