News for hackers. Stuff that matters.
Re:Introducing TypeChat from Microsoft by geysersam on Friday July 21, 2023 @03:08AM (#36810858)
Well, if the output doesn't conform to the format it's useless. If the model can't produce good and correct output then it's simply not up to the task.
Re:Introducing TypeChat from Microsoft by IanCal on Friday July 21, 2023 @04:27AM (#36811417)
That really strongly depends on your task. Lots of tasks can accept a non-zero failure rate in return for better results on the successful cases. I'm not sure I can think of any off the top of my head where you'd use a LLM and can never deal with a failure, particularly if you're using an external service where you're guaranteed to have to deal with errors or downtime at some point.
Re:Introducing TypeChat from Microsoft by waffletower on Friday July 21, 2023 @01:58PM (#36817164)
In my experience, LLM responses result in a fair distribution of outputs that do have semantically useful outputs but do not precisely adhere to the requested format. If I chose to use a strongly typed language for LLM parsing, perhaps I would be tempted to eliminate complexity and simply throw structural outliers away, and explain to the suits that a certain percentage of our queries/expenses are unusable. Instead, more sophisticated coercion techniques could be applied instead to increase output utilization.