There are tools that you can use to force the model to give structured output, such as llama.cpp GBNF grammar for example. They're a bit harder to use than ask gpt4 but they do work pretty well for what I use it for.