Re:Introducing TypeChat from Microsoft
by geysersam on Friday July 21, 2023 @03:08AM
(#36810858)
Well, if the output doesn't conform to the format it's useless. If the model can't produce good and correct output then it's simply not up to the task.
[ Parent
| on HN ]
That really strongly depends on your task. Lots of tasks can accept a non-zero failure rate in return for better results on the successful cases. I'm not sure I can think of any off the top of my head where you'd use a LLM and can never deal with a failure, particularly if you're using an external service where you're guaranteed to have to deal with errors or downtime at some point.
Re:Introducing TypeChat from Microsoft
by waffletower on Friday July 21, 2023 @01:58PM
(#36817164)
In my experience, LLM responses result in a fair distribution of outputs that do have semantically useful outputs but do not precisely adhere to the requested format. If I chose to use a strongly typed language for LLM parsing, perhaps I would be tempted to eliminate complexity and simply throw structural outliers away, and explain to the suits that a certain percentage of our queries/expenses are unusable. Instead, more sophisticated coercion techniques could be applied instead to increase output utilization.