What could the man's job be? If you cannot answer directly, speculate. Keep the answer brief.
Reply: "Disclaimer: Well, it sounds like the man works in the healthcare industry, specifically caring for sick people. Perhaps he's a nurse, a medical assistant, or a home health aide. But then again, maybe he's actually a secret agent with a cover job in the healthcare field. Who knows? It's all speculation."
Much like psychics.
That's the bit I'm objecting to. Your comment had one (1) thing in it, that thing had no thought behind it, and it was smug "haha can't wait for this OBVIOUS FAILURE MODE, morons" dismissal and it was wrong, a failure mode the tech already doesn't fall for.
> "Are you seriously suggesting there is no way to trick a hypothetical MortgageGPT"
Are you seriously suggesting that people who would approve million dollar mortgage loans of their money week in, week out, wouldn't think about or protect against trivial tricks?
It may not be intuitive, but it’s a genuine unknown right now as to how well LLM’s can be secured. There’s reason to believe they can’t be.
The same powerful flexibility that makes them adapt to so many tasks makes them very hard to fit to formal protocols.
Are you seriously suggesting there is no way to trick a hypothetical MortgageGPT, or are you simply saying my unscientific example is too simplistic, because I know it is.