GPT-4 Outperforms Elite Crowdworkers, Saving Researchers $500k and 20k hours

Re:GPT-4 Outperforms Elite Crowdworkers, Saving Researchers ... by rossdavidh on Tuesday April 11, 2023 @10:08PM (#35534212)

So, uh, GPT-4 outperforms at labeling. What is that labeling used for?

"Employing Surge AI's top-tier human annotators at a rate of $25 per hour would have cost $500,000 for 20,000 hours of work, an excessive amount to invest in the research endeavor. Surge AI is a venture-backed startup that performs the human labeling for numerous AI companies including OpenAI, Meta, and Anthropic."

What could go wrong? Using GPT-4 to perform labeling used by OpenAI in order to train...uh, wait.

[ Parent | on HN ]

Re:GPT-4 Outperforms Elite Crowdworkers, Saving Researchers ... by rossdavidh on Tuesday April 11, 2023 @10:09PM (#35534216)

https://en.wikipedia.org/wiki/Positive_feedback

[ Parent | on HN ]

Re:GPT-4 Outperforms Elite Crowdworkers, Saving Researchers ... by mztwo on Tuesday April 11, 2023 @10:14PM (#35534250)

Yep - you highlighted exactly what raised my eyebrow as I was writing the article.

[ Parent | on HN ]

Re:GPT-4 Outperforms Elite Crowdworkers, Saving Researchers ... by nethdeco on Wednesday April 12, 2023 @03:20AM (#35536077)

And the noise would keep adding up.

[ Parent | on HN ]

Re:GPT-4 Outperforms Elite Crowdworkers, Saving Researchers ... by mclightning on Wednesday April 12, 2023 @04:15AM (#35536459)

This is a bigger problem than people realise.

Think about it, how many millions of articles are posted online produced by OpenAI's GPTs to date... Good luck clearing out the training data for GPT-5.

True human content will get gradually scarce. We steer it for sure for our posts, but it is still GPTs that do the heavy lifting.

OpenAI's own classifier fails to detect GPT-4 generated text at the moment.

[ Parent | on HN ]

Re:GPT-4 Outperforms Elite Crowdworkers, Saving Researchers ... by pixl97 on Wednesday April 12, 2023 @03:19PM (#35545244)

>classifier fails to detect GPT-4 generated text

That's because beyond the 'As an AI language model' and a few key words it can be nearly impossible to detect GPT-4 especially if any prompt is used to intentionally keep it from being detected.

Human like text is a solved problem. There is no more getting better at detecting AI written text, there is only classifying more humans incorrectly at this point.

[ Parent | on HN ]

Re:GPT-4 Outperforms Elite Crowdworkers, Saving Researchers ... by realusername on Wednesday April 12, 2023 @03:11PM (#35545118)

We're also using computers to build better computers, it makes sense

[ Parent | on HN ]