Re:Every Way to Get Structured Output from LLMs
by hellovai on Tuesday June 18, 2024 @02:04AM
(#40714509)
Hey everyone! One of the creators of BAML here! Appreciate sharing this post. For anyone interested in playing around with an interactive version of BAML online, check it out here: https://www.promptfiddle.com
[ Parent
| on HN ]
Re:Every Way to Get Structured Output from LLMs
by dsign on Tuesday June 18, 2024 @03:02AM
(#40714868)
Really interesting library! In the docs, could you describe in a bit more detail which kind of JSON errors it tolerates? And which models currently work best with your parsing approach?
Re:Every Way to Get Structured Output from LLMs
by hellovai on Tuesday June 18, 2024 @03:13AM
(#40714932)
Thanks! We should add that to the docs haha. But the here's a few:
- keys without strings
- coercing singular types -> arrays when the response requires an array
- removing any prefix or suffix tags
- picking the best of many JSON candidates in a string
- unescaped newlines + quotes so "afds"asdf" converts to "afds\"asdf"
In terms of models, honestly, we tried as bad as llama2, and it seems to work in quite a few use cases