Re:AI and the Problem of Knowledge Collapse
by FridgeSeal on Friday April 05, 2024 @10:26PM
(#39949488)
But this is _exactly_ the problem being discussed.
Domain expert: “I cannot get it to produce something that even approximately resembles x”
Person: “I gave it the paper and I got maybe x?”
There is a fundamental knowledge gap here, just because you can get it to do something isn’t the goal, or even a good outcome. The illusion that it’s capable of doing something it evidently cannot is a dangerous one, because people who don’t know better, and are being convinced that they do understand or comprehend when they don’t.
Domain expert: “I cannot get it to produce something that even approximately resembles x”
Person: “I gave it the paper and I got maybe x?”
There is a fundamental knowledge gap here, just because you can get it to do something isn’t the goal, or even a good outcome. The illusion that it’s capable of doing something it evidently cannot is a dangerous one, because people who don’t know better, and are being convinced that they do understand or comprehend when they don’t.