News for hackers. Stuff that matters.
Re:Speech and Language Processing (3rd ed. draft) by chaxor on Monday March 11, 2024 @12:36PM (#39670338)
If you want to process 100M documents, you want to use the most performant and fastest option - which a 7b param model isn't going to be it.

Additionally, entity linking is an extremely common task, for which LLMs fail at pretty miserably (certainly if the dictionary is custom/private. Additional work must be done to somehow (!? Many options here ?!) perform EL.

So, in the end, making a silver corpus from an LLM may be an option for NER to train a much much smaller algorithm. But EL is _still_ not a 'plug and play' problem, and can actually be pretty difficult to do "well" (using the modern techniques of MHS, etc).

There are no comments on this comment yet.