News for hackers. Stuff that matters.

A comment on LLMs can label data as well as human annotators, but 20 times faster. Parent comment

Re:LLMs can label data as well as human annotators, but 20 t... by natch on Sunday June 18, 2023 @05:38PM (#36384778)
* * *
[ Parent | on HN ]
Re:LLMs can label data as well as human annotators, but 20 t... by nihit-desai on Sunday June 18, 2023 @05:52PM (#36384902)
I mean, sure. For ground truth, we are using the labels that are part of the original dataset: * https://huggingface.co/datasets/banking77 * https://huggingface.co/datasets/lex_glue/viewer/ledgar/train * https://huggingface.co/datasets/squad_v2 ... (exhaustive set of links at the end of the report).

Is there some noise in these labels? Sure! But the relative performance with respect to these is still a valid evaluation

[ Parent | on HN ]
Re:LLMs can label data as well as human annotators, but 20 t... by natch on Sunday June 18, 2023 @10:53PM (#36386826)
Agreed, thanks for highlighting these links!
[ Parent | on HN ]