Re:We’re receiving about 3,000 reports/hour
by pfisherman on Saturday November 16, 2024 @05:36PM
(#42160007)
Why use an LLM as opposed to a more narrow purpose built model? LLMs are not beating smaller, purpose built models on tasks like POS tagging, NER, sentiment analysis, etc. And the inference costs scale quite poorly (unless you are self hosting llama or something).
[ Parent
| on HN ]
Re:We’re receiving about 3,000 reports/hour
by OutOfHere on Saturday November 16, 2024 @05:57PM
(#42160217)
That's where "rapidly" comes in. Also, LLMs allow very high customization via the choice of prompt. It's a lot quicker to adapt the prompt than to retrain a fine-tuned model. I think the outputs of the stabilized LLM could later be used to properly fine-tune a custom model for efficient use.
As for sentiment, even embeddings can do a good job at it.