"deeply sophisticated and constrained requirements"

Yes this resonates completely. I think many are forgetting the purpose of formal language and code was because natural language has such high ambiguity that it doesn't capture complex behavior

LLMs are great at interpolating between implicit and unsaid requirements but whether their interpolation matches your mental model is a dice throw