News for hackers. Stuff that matters.
Re:A guidance language for controlling LLMs by simonw on Tuesday May 16, 2023 @01:59PM (#35965426)
This is pretty fascinating, but I'm not sure I understand the benefit of using a Handlebars-like DSL here.

For example, given this code from https://github.com/microsoft/guidance/blob/main/notebooks/ch...

    create_plan = guidance('''{{#system~}}
    You are a helpful assistant.
    {{~/system}}
    {{#block hidden=True}}
    {{#user~}}
    I want to {{goal}}.
    {{~! generate potential options ~}}
    Can you please generate one option for how to accomplish this?
    Please make the option very short, at most one line.
    {{~/user}}
    {{#assistant~}}
    {{gen 'options' n=5 temperature=1.0 max_tokens=500}}
    {{~/assistant}}
    {{/block}}
    {{~! generate pros and cons and select the best option ~}}
    {{#block hidden=True}}
    {{#user~}}
    I want to {{goal}}.
    ''')
How about something like this instead?

    create_plan = guidance([
        system("You are a helpful assistant."),
        hidden([
            user("I want to {{goal}}."),
            comment("generate potential options"),
            user([
                "Can you please generate one option for how to accomplish this?",
                "Please make the option very short, at most one line."
            ]),
            assistant(gen('options', n=5, temperature=1.0, max_tokens=500)),
        ]),
        comment("generate pros and cons and select the best option"),
        hidden(
            user("I want to {{goal}}"),
        )
    ])
Re:A guidance language for controlling LLMs by marcotcr on Tuesday May 16, 2023 @02:15PM (#35965601)
I think the DSL is nice when you want to take part of the generation and use it later in the prompt, e.g. this (in the same notebook).

---

prompt = guidance('''{{#system~}}

You are a helpful assistant.

{{~/system}}

{{#user~}}

From now on, whenever your response depends on any factual information, please search the web by using the function <search>query</search> before responding. I will then paste web results in, and you can respond.

{{~/user}}

{{#assistant~}}

Ok, I will do that. Let's do a practice round

{{~/assistant}}

{{>practice_round}}

{{#user~}}

That was great, now let's do another one.

{{~/user}}

{{#assistant~}}

Ok, I'm ready.

{{~/assistant}}

{{#user~}}

{{user_query}}

{{~/user}}

{{#assistant~}}

{{gen "query" stop="</search>"}}{{#if (is_search query)}}</search>{{/if}}

{{~/assistant}}

{{#if (is_search query)}}

{{#user~}}

Search results: {{#each (search query)}}

<result>

{{this.title}}

{{this.snippet}}

</result>{{/each}}

{{~/user}}

{{#assistant~}}

{{gen "answer"}}

{{~/assistant}}

{{/if}}''')

---

You could still write it without a DSL, but I think it would be harder to read.

Re:A guidance language for controlling LLMs by slundberg on Tuesday May 16, 2023 @02:08PM (#35965532)
You can serialize and ship the DSL to a remote server for high speed execution. (without trusting raw Python code)
Re:A guidance language for controlling LLMs by foota on Tuesday May 16, 2023 @02:30PM (#35965763)
There's prior art for pythonic DSLs that aren't actual python code.
Re:A guidance language for controlling LLMs by crooked-v on Tuesday May 16, 2023 @03:21PM (#35966434)
Why not just use JSON instead, though? Then you can just rely on all the preexisting JSON tooling out there for most stuff to do with it.
Re:A guidance language for controlling LLMs by itake on Tuesday May 16, 2023 @02:05PM (#35965495)
My guess is you can store the DLS as a file (or in a db). With your example, you have to execute the code stored in your db.
Re:A guidance language for controlling LLMs by PeterisP on Tuesday May 16, 2023 @04:18PM (#35967196)
Your example assumes a nested, hierarchical structure while the former example is strictly linear. IMHO that's the key difference there, as the former can (and AFAIK is) be directly encoded and passed to the LLM, which inherently receives only a flat list of tokens.

Your example might be nicer to edit, but then it would still have to be translated to the actual 'guidance language' which would have to look (and be) flat.

Re:A guidance language for controlling LLMs by emehex on Tuesday May 16, 2023 @04:50PM (#35967588)
We could write a python package that could? A codegen tool that generates codegen that will then generate code? <insert xzibit meme here>
Re:A guidance language for controlling LLMs by netdur on Tuesday May 16, 2023 @05:15PM (#35967841)
I think chatgpt4 can easily write the python code... wait a second!
Re:A guidance language for controlling LLMs by quickthrower2 on Tuesday May 16, 2023 @08:03PM (#35969516)
Would love to hear your opinion on guidance, in the context of prompt injection attacks :-)