lamma.cpp allows you to restrict the output such that it would always generate valid JSON https://github.com/ggerganov/llama.cpp#constrained-output-wi...