Inversion: Fast, Reliable Structured LLMs

Re:Inversion: Fast, Reliable Structured LLMs by dartos on Tuesday March 19, 2024 @08:05AM (#39754889)

You can force LLMs to generate valid json by using a context free grammar FWIW

[ Parent | on HN ]

Re:Inversion: Fast, Reliable Structured LLMs by zuck_vs_musk on Tuesday March 19, 2024 @11:55PM (#39762734)

Please elaborate

[ Parent | on HN ]

Re:Inversion: Fast, Reliable Structured LLMs by kaiokendev on Wednesday March 20, 2024 @08:34AM (#39765545)

Matt Rickard has a good entry level blog post about it, from the angle of regex constraining [0]. Context free grammars follow the same principle, except using a finite state machine to restrict the action space.

[0]: https://matt-rickard.com/rellm

[ Parent | on HN ]

Re:Inversion: Fast, Reliable Structured LLMs by dartos on Thursday March 21, 2024 @05:34PM (#39784675)

Look at llama.cpp grammars, lmql, or guidance-ai

[ Parent | on HN ]