Replace LLM with "clerk" and "generated case notes" with "copy pasted from my standard legally approved phrasing" playbook. Government workers who fill out standard reports already have standard forms pre filled with the results they know they are landing out, with certain sections that differ ready to be edited while the rest stays the same, look at things like warrants as an example. When I worked a government job i was literally handed templates by my supervisor of "pre approved ways to phrase things", that they percieved would help avoid lawsuits or any contest.
With the LLM, everyone will clutch their pearls and be shocked.
At an abstract level, this sounds like a precursor to war. Everyone has competing interests and the peaceful way to resolve them is communication. When that stops force starts.
Maybe the out is that there is some automated sense of listening and agreement.
Instead what we got instead are surveillance capitalism and now this LLM/AI stuff that becomes much better are emulating human behaviour and languages. UX in software has NOT improved, and I dare to say it has gotten worse. Just feel the performance of iOS 6 on iPod touch vs the lates iOS. Tech over quarter century has just tech getting more complex, and made more easily to centralised and consolidate control. Is this what we really want?
The problem is that we’re going to have people litigating court cases and stupid bureaucratic nonsense within companies by lobbing nonsense that nobody has read at each other. Everyone will wink and nod, but these organizations and institutions will be exponentially more clueless.
As technologists, it’s easy to shrug. What do you do when a judge takes your kid away after using an LLM to read and interpret bullshit generated case notes that a CPS worker generated with another LLM?