Yep, that's also a huge issue with LLMs in production. Our product has some automatic detection of jailbreak attempts so that you can see when people are trying to jailbreak & prompt inject, but hallucinations are the biggest unsolved problem imo.