Re:Advanced NLP with SpaCy
by pletnes on Saturday September 09, 2023 @07:26AM
(#37444153)
Good points. This means you can use LLMs to create an 80% accurate dataset, then manually correct the last 20% to get all the way.
[ Parent
| on HN ]
Re:Advanced NLP with SpaCy
by john-radio on Saturday September 09, 2023 @07:29AM
(#37444165)
"Manually" anything just depends on the size of your dataset.
Manually correcting the wrong 20% may be a reasonable amount of work, but you must examine all 100 to find the 20 that needs fixing. And that is most likely not a reasonable amount of work.
I’ve done image annotation for ML / computer vision. I found it extremely useful to use my first annotations to train a poor model, then use results from that to annotate new data. You get feedback on model quality as you go, and looking at 100 images with multiple objects is way less work than annotating 100 images.
If it’s worth it? I guess that depends on the project you’re working on.