News for hackers. Stuff that matters.
Re:The Rise of the AI Engineer by z3c0 on Friday June 30, 2023 @05:52PM (#36542875)
No. No they are not. Do not keep proliferating this idea, as there are real consequences. Your input only informs the likelihood of the output. There is no model actually extrapolating rules from the information, so the premise of LLMs being extractive is 100% verifiably false. They summarize well due to being handed a roughly correct arrangement of tokens to mimic the order of. This should not be confused with extraction.

To put it simply, keep falling for that, and it'll bite you in the ass.

Re:The Rise of the AI Engineer by visarga on Saturday July 01, 2023 @12:31AM (#36546605)
You are talking from a top-down point of view. Of course they can make an error any time, after all they stochastically sample the output. All approaches to information extraction are upper bounded at 90-95% accuracy, I have extensive experience with this task and run many evaluations on invoices, receipts, forms and other doctypes. Human in the loop is still required.

But in practice you can rely on good copying and excellent ability to parse names, addresses and various values as good or better than other approaches. LLMs reduce the number of corrections, and they are easier to deploy - instead of labeling hundreds of documents you can just query with the field name.

Re:The Rise of the AI Engineer by z3c0 on Saturday July 01, 2023 @04:35PM (#36554887)
* * *
Re:The Rise of the AI Engineer by pvman on Wednesday July 05, 2023 @05:37AM (#36597959)
Thanks for the info z3c0. Have you implemented open source solutions? Do you mind sharing some deterministic solutions to detect entities without huge dictionaries (NER)? Or maybe extractive QA if you used that instead?
Re:The Rise of the AI Engineer by z3c0 on Wednesday July 05, 2023 @02:27PM (#36604860)
Deterministic? Not anything that is "one size fits all". If your documents are of an unpredictable shape, then ML is your best bet. For both NER and QA, BERT models are very capable.

To your first question, most of my ML work is proprietary, unfortunately. I am hoping to change that in the near future.

Edit: spaCy is a great library for NER, if you're hoping for an open-source solution that can have you hitting the ground quickly. SQuAD for QA.