Re:ChatGPT's API Is So Good and Cheap, It Makes Most Text Ge...
by celestialcheese on Saturday March 11, 2023 @11:11PM
(#35116579)
For sure. If perfect accuracy is important, it's still good to do sampling and human review to figure out accuracy rate and decide if further checking is required. But it still beats the pants off accuracy of other methods for the amount of work required.
That said, with fine-tuning, `davinci-003` is _excellent_ at the types of entity extraction you're describing.
Re:ChatGPT's API Is So Good and Cheap, It Makes Most Text Ge...
by dandiep on Sunday March 12, 2023 @12:39AM
(#35117056)
There is no such thing as davinci-003 per se. There is text-davinci-003 which you can't fine tune, and davinci which you can.
Re:ChatGPT's API Is So Good and Cheap, It Makes Most Text Ge...
by rolisz on Sunday March 12, 2023 @07:31AM
(#35118908)
Sure, but SOTA on NER is around 90% for things like Names and Places. For skills, a Spacy model will get around 60-70%. And training a Spacy model takes a bit of fiddling.
Example, I tried to extract skills from a job posting. ChatGPT did well, but there were skills missing.
It is good to find some entities but then you need to extend the labeling manually.