News for hackers. Stuff that matters.
Re:Every Way to Get Structured Output from LLMs by hellovai on Tuesday June 18, 2024 @02:04AM (#40714509)
Hey everyone! One of the creators of BAML here! Appreciate sharing this post. For anyone interested in playing around with an interactive version of BAML online, check it out here: https://www.promptfiddle.com
Re:Every Way to Get Structured Output from LLMs by dsign on Tuesday June 18, 2024 @03:02AM (#40714868)
Really interesting library! In the docs, could you describe in a bit more detail which kind of JSON errors it tolerates? And which models currently work best with your parsing approach?
Re:Every Way to Get Structured Output from LLMs by hellovai on Tuesday June 18, 2024 @03:13AM (#40714932)
Thanks! We should add that to the docs haha. But the here's a few:

- keys without strings

- coercing singular types -> arrays when the response requires an array

- removing any prefix or suffix tags

- picking the best of many JSON candidates in a string

- unescaped newlines + quotes so "afds"asdf" converts to "afds\"asdf"

In terms of models, honestly, we tried as bad as llama2, and it seems to work in quite a few use cases

Re:Every Way to Get Structured Output from LLMs by dsign on Tuesday June 18, 2024 @03:17AM (#40714962)
Thanks! I see myself using the library soon :-)