News for hackers. Stuff that matters.
Re:OpenAI Quietly Shuts Down Its AI Detection Tool by frumper on Tuesday July 25, 2023 @11:57AM (#36864257)
The AI tool doesn't give accurate results. You don't know when it's not accurate. There is no accurate way to check its results. Who should use a tool to help them make a decision when you don't know when the tool will be wrong and it has a low rate of accuracy? It's in the article.
Re:OpenAI Quietly Shuts Down Its AI Detection Tool by bgirard on Tuesday July 25, 2023 @12:22PM (#36864780)
> The AI tool doesn't give accurate results.

Nearly everything doesn't give 100% accurate results. Even CPUs have had bugs their calculation. You have to use a suitable tool for a suitable job with the correct context while understanding it's limitation to apply it correctly. Now that is proper engineering. You're partially correctly but you're overstating:

> A tool that gives incorrect and inconsistent results shouldn’t have any part of a decision making process.

That's totally wrong and an overstated position.

A better position is that some tools have such a low accuracy rate that they shouldn't be used for their intended purpose. Now that position I agree with it. I accept that CPUs may give incorrect results due to a cosmic ray event, but I wouldn't accept a CPU that gives the wrong result for 1/100 instructions.

Re:OpenAI Quietly Shuts Down Its AI Detection Tool by frumper on Tuesday July 25, 2023 @12:26PM (#36864863)
The thread is about tools to evaluate LLMs. Please re-read my comment in that light and generously assume I'm talking about that.
Re:OpenAI Quietly Shuts Down Its AI Detection Tool by __loam on Tuesday July 25, 2023 @01:23PM (#36865918)
Your comment applies to all these tools though lol. No need to clarify, it's all a probabilistic machine that's very unreliable.