News for hackers. Stuff that matters.
Re:A guidance language for controlling LLMs by rcarmo on Tuesday May 16, 2023 @02:43PM (#35965922)
I'm getting valid JSON out of gpt-3.5-turbo without trouble. I supply an example via the assistant context, and tell it to output JSON with specific fields I name.

It does fail roughly 1/10th of the time, but it does work.

Re:A guidance language for controlling LLMs by harshhpareek on Tuesday May 16, 2023 @05:59PM (#35968326)
10% failure rate is too damn high for a production use case.

What production use case, you ask? You could do zero-shot entity extraction using ChatGPT if it were more reliable. Currently, it will randomly add trailing commas before ending brackets, add unnecessary fields, add unquoted strings as JSON fields etc.

Re:A guidance language for controlling LLMs by rcarmo on Wednesday May 17, 2023 @09:21AM (#35974883)
Which is why this is just an experiment. I’ve gone back to standard translation APIs for everything except the final summarizing (and even them I might go there as well).