Re:How to talk to someone who doesn't trust AI
by Terr_ on Thursday April 25, 2024 @04:38PM
(#40162696)
To me there's an intermediate and more-immediate kind of distrust going on: Everyone should distrust and avoid any tool which unpredictably fails in serious ways that are hidden from the user.
For example, if you were digitizing important blueprints and financial/legal documents before trashing the originals, but the scanning process was unpredictably changing numbers inside text [0] so that your copies were wrong in small but incredibly serious ways.
In the case of LLMs (or the example scanner) the tool will eventually yield answer which looks correct but is dangerously wrong, and the amount of continuous human fact-checking--itself another form of distrust!--needed can be so high that any labor-savings are wiped out.
For example, if you were digitizing important blueprints and financial/legal documents before trashing the originals, but the scanning process was unpredictably changing numbers inside text [0] so that your copies were wrong in small but incredibly serious ways.
In the case of LLMs (or the example scanner) the tool will eventually yield answer which looks correct but is dangerously wrong, and the amount of continuous human fact-checking--itself another form of distrust!--needed can be so high that any labor-savings are wiped out.
[0] http://www.dkriesel.com/en/blog/2013/0802_xerox-workcentres_...