Yes it's true that LLMs hallucinate facts, but there are ways to control that. Despite the challenges they can spit out perfectly functional code to spec to boot. So for me it's not too much of a stretch to think that it'd do a reasonably good job at defending simple cases.