>you should also make a rule saying astrology books aren't a legitimate source of sentence guidance
My example was absurd on purpose, I didn't want to bring an example where people could respond with "well, actually..."
But in the real world that is rarely the case, imagine substituting "astrology book" for "Bible/Quran/..." Would that be considered a legitimate source of sentence guidance? I'm sure people would spend years arguing about that...
As a society we need to understand that LLM hallucination is no different than a bloom filter giving you a false-positive.
Then I guess I don't understand the point you are trying to make. Are you saying that the problem is unsolvable and people should just accept that?
My point is that LMM is a tool, and we should never attribute any sort of blame to our tools. The blame should always be with the humans.
In your example you should probably drop the judge, but you should also make a rule saying astrology books aren't a legitimate source of sentence guidance. That's what people are annoyed about re:LLMs. People keep insisting they are a legit source in different situations.
You wouldn't ban them overall, but you do want some kind of society-level taboo against relying on them. You can't just deal with it on the level of people who get fooled into using them.