Regardless of what changes the bank makes, it’s not going to let you access someone else’s money. This llm very well might.

Well, software has been known to have vulnerabilities...

Consider this: the bank teller is non-deterministic, too. They could give you 500 dollars of someone else's money. But they don't, generally.

Bank tellers are deterministic though. They have a set protocol for each cases and escalate unknown cases to a more deterministic point of contact.

It will be difficult to incorporate relative access or restrictions to features with respect to users current/known state or actions. Might as well write the entire web app at that point.

I think the bank teller's systems and processes are deterministic, but the teller itself is not. They could even rob the bank, if they wanted to. They could shoot the customers. They don't, generally, but they can.

I think, if we can efficiently capture a way to "make" LLMs conform to a set of processes, you can cut out the app and just let the LLM do it. I don't think this makes any sense for maybe the next decade, but perhaps at some point it will. And, in such time, software engineering will no longer exist.

The actual app is the set of processes.