If there was a hammer that, held one way, drove nails perfectly in one blow, and held another, made it look like the nail was driven but actually broke it right at the junction so that the work pieces weren't fastened... I'd say that the second way was the wrong way to use that hammer.

Not only wrong, but dangerous, because nails are often used to fasten structural elements of houses, and incorrect but hard-to-detect flaws like this could result in collapse.

Similarly, if ChatGPT gives you an answer high in truthiness but low in accuracy, it could negatively impact you, whether loss in credibility if you repeat nonsense in front of someone knowledgable, or even worse if you use the incorrect knowledge to try to solve a real world problem.

The issue is that approach maximizes its flaws. It’s like shooing yourself in the foot with a nail gun, that’s objectively the wrong thing to do.

It's a tool whose flaw is that sometimes it secretly does the exact opposite of what you're trying to do with it in a way that's only detectable if you didn't already need it.

It's like Babelfish, but for monoglots.