Re:LLMs can label data as well as human annotators, but 20 t...
by nihit-desai on Sunday June 18, 2023 @05:35PM
(#36384749)
Trying to parse your comment - if I understand correctly, you're suggesting LLMs will play a big role in labeling data in the future?
I agree, but for a different reason - they're better at labeling data compared to human annotators. On quality, cost and turnaround time. FWIW, i think it's not either/or - LLMs will be leveraged extensively but with supervision, as a co-pilot.
Re:LLMs can label data as well as human annotators, but 20 t...
by lumost on Sunday June 18, 2023 @07:07PM
(#36385466)
Have you worked with labeling services before? The quality is always terrible. At least with an LLM I can get consistently terrible output, quickly.
Re:LLMs can label data as well as human annotators, but 20 t...
by klyrs on Monday June 19, 2023 @12:17AM
(#36387284)
A few years ago, I started spoiling answers to captchas. I make a game of getting as many false positives and negatives I can get away with.
Re:LLMs can label data as well as human annotators, but 20 t...
by rossdavidh on Monday June 19, 2023 @06:49PM
(#36397409)
"Consistently terrible output, quickly!"(TM)
1) it will work well, at first, and only become low-quality after they (and their budgets) have become accustomed to paying 1/20th as much for the service
2) even if they pay for "human" labeling, they will go for the low cost bid, in a far-away country, which will subcontract to an LLM service without telling them
3) "hey, we should pay more for this input, in order to avoid not-yet-seen quality problems in the future", has practically never won an argument in any large corporation ever. I won't say absolutely 0 times, but pretty close.
Long story short, the use of LLM's by Big Tech may be doomed. Much like how "SEO optimization" turns quickly into clickbait and link farms if there is not high-urgency and high-priority efforts to combat it, LLM's (and other trendy forms of AI that require lots of labeled input) will quickly turn sour and produce even less impressive results than they already do.
The current wave of "AI" hype looks set to succeed about as well as IBM Watson.