News for hackers. Stuff that matters.

marcotcr

Re:ChatGPT vs. open source on harder tasks on Friday May 19, 2023 @10:25AM (#36002373)
One of the authors here.

I deleted the notebook yesterday after realizing it was weird to have a disclaimer saying 'this is our personal opinion' on a github repo under 'Microsoft'. I moved the notebook to a gist here: - https://gist.github.com/marcotcr/64ca85bd0be724f6d8fb8f1b3d2...

I didn't know we were on hackernews, otherwise I would of course not have made it a broken link :)

This morning, we put the link back, without the takeaways with our personal opinion at the end (it's still on the gist and on the blog post https://medium.com/@marcotcr/exploring-chatgpt-vs-open-sourc... )

Re:A guidance language for controlling LLMs on Tuesday May 16, 2023 @02:15PM (#35965601)
I think the DSL is nice when you want to take part of the generation and use it later in the prompt, e.g. this (in the same notebook).

---

prompt = guidance('''{{#system~}}

You are a helpful assistant.

{{~/system}}

{{#user~}}

From now on, whenever your response depends on any factual information, please search the web by using the function <search>query</search> before responding. I will then paste web results in, and you can respond.

{{~/user}}

{{#assistant~}}

Ok, I will do that. Let's do a practice round

{{~/assistant}}

{{>practice_round}}

{{#user~}}

That was great, now let's do another one.

{{~/user}}

{{#assistant~}}

Ok, I'm ready.

{{~/assistant}}

{{#user~}}

{{user_query}}

{{~/user}}

{{#assistant~}}

{{gen "query" stop="</search>"}}{{#if (is_search query)}}</search>{{/if}}

{{~/assistant}}

{{#if (is_search query)}}

{{#user~}}

Search results: {{#each (search query)}}

<result>

{{this.title}}

{{this.snippet}}

</result>{{/each}}

{{~/user}}

{{#assistant~}}

{{gen "answer"}}

{{~/assistant}}

{{/if}}''')

---

You could still write it without a DSL, but I think it would be harder to read.

Re:A guidance language for controlling LLMs on Tuesday May 16, 2023 @01:55PM (#35965369)
We're biased, but we think guidance is still very useful even with OpenAI models (e.g. in https://github.com/microsoft/guidance/blob/main/notebooks/ch... we use GPT-4 to do a bunch of stuff). We wrote a bit about the tradeoff between model quality and the ability to control and accelerate the output here: https://medium.com/p/aa0395c31610