“Verify all facts and compliance requirements” leaves enormous holes even if you assume the LLM has a concept of facts and requirements (it does not).
What facts? What requirements? For what industry? For what subset of that industry? For what country or countries that you will be doing business in? Are these current “facts” and “requirements” or is the LLM referencing a dusty article from 1992 for which the subject matter has been radically overhauled?
In my job I regularly see small but incredibly important mistakes like this lead to major issues. Some of those are human driven but increasingly the defense of the person responsible has turned into “Claude said it was fine though!”
If I plucked a random passerby and gave them the task then no, I’d find myself detailing out every specific to them.
You’re equating the LLM to the least qualified candidates. I don’t think your argument is communicating what you intended.
feel like the parent you are replying to literally views their place of work as a daycare which is very condescending
The argument is regarding LLMs and domain knowledge.
Additionally, using a specific tool does not suddenly give the model common sense enough to say “this piece of information doesn’t answer the question of whether this solution fits in this specific industry at this time in this place”.
No. This is a disasterous instruction. Not only is it vague, but it's also meaningless. When giving instructions to an LLM your prompt must be concise and exact. Tell it _exactly_ which requirements need to be followed, ideally have it write or (preferably) pass audited tests to enforce these requirements. You also need to provide it with a hard source of truth it can rely upon. Instead of saying "verify facts", you're better off by saying "... make sure [whatever you're doing] matches with data at X.Y.Z, verify by running [instruction/command/program]"
I remember hearing that 10 years ago about self-driving.
And the thing he complained about is fixable with a web search, and AI does programming and office work today. So, it's already here. It's just a question of degrees.
Tesla has been a couple years away from FSD for, what, like ten years now?
If you scrape off the glitter, you'll find a lot more duct tape and wire than you think.
We need a lot more basic research into LLMs and also a lot cheaper hardware.
The current batch of LLMs will turn a lot of fields upside down, but not to the tune of $3tn or whatever crazy amounts are being invested right now.
IME people would benefit greatly from the process, albeit tedious and time-consuming, of testing out the same prompt sequence/session with the exact same model multiple times. It becomes clear extremely quickly how capable but unreliable and inconsistent a model can be even when given the same context. If you have ever completed a long, complicated task with an agent and then lost the session and tried doing the same thing again from scratch you may have had the experience of seeing the subtle changes that come up in the model's thinking which lead it to accept or reject certain paths and ignore or incorporate prompt instructions like the one you've provided.
But really that particular issue could have been solved by literally just telling it in a markdown file or instructions something like "verify all facts or compliance requirements with web search and include citations in responses".