LLMs can label data as well as human annotators, but 20 times faster

Re:LLMs can label data as well as human annotators, but 20 t... by orangepurple on Sunday June 18, 2023 @04:39PM (#36384228)

This will probably work as long as the material being annotated is similar to the material the LLM was trained on. When it encounters novel data (value) it will likely perform poorly.

[ Parent | on HN ]

Re:LLMs can label data as well as human annotators, but 20 t... by nihit-desai on Sunday June 18, 2023 @05:08PM (#36384498)

Partially agree, but it's a continuous value rather than a boolean. We've seen LLM performance largely follow this story: https://twitter.com/karpathy/status/1655994367033884672/phot...

From benchmarking, we've been positively surprised by how effective few-shot learning and PEFT are, at closing the domain gap.

"When it encounters novel data (value) it will likely perform poorly" -- is that not true of human annotators too? :)

[ Parent | on HN ]

Re:LLMs can label data as well as human annotators, but 20 t... by orangepurple on Sunday June 18, 2023 @10:30PM (#36386686)

> "When it encounters novel data (value) it will likely perform poorly" -- is that not true of human annotators too? :)

Some humans have intelligence and reasoning abilities. No LLMs do :)

[ Parent | on HN ]

Re:LLMs can label data as well as human annotators, but 20 t... by yawnxyz on Tuesday June 20, 2023 @08:33PM (#36412769)

I always regret making that assumption

[ Parent | on HN ]