For a less dramatic strategy with LLMs that expose the tokenizer vocabulary, you can use context-free grammars to constrain the logits according to the parser so that the LLMs only generate valid next tokens for the language.[0]

[0]https://github.com/r2d4/parserllm