Pay a bunch of people to go through and index your book collection and you'll get some errors too.
What's interesting about LLMs is they take tasks that were previously impossible - I'm not going to index my book collection, I do not have the time or willpower to do that - and turned them into things that I can get done to a high but not perfect standard of accuracy.
I'll take a searchable index of my books that's 95% accurate over no searchable index at all.
I'm currently building out some code that should go in production in the next week or two and simply because of this we are using LLM to prefill data and then have a human look over it.
For our use case the LLM prefilling the data is significantly faster but if it ever gets to the point of that not needing to happen it would take a task whichtakes about 3 hours ( now down to one hour ) and make it a task that takes 3 minutes.
Will LLMs ever get to the point where it is perfectly reliable ( or at least with an error margin low enough for our use case ), I don't think so.
It does make for a very cheap accelerator though.
So, even if data entry is incredibly fast, curation is still time-consuming. On balance, would it even be faster to capture the ISBN code of 100 books with a scanner app, assuming that the index lookup is correct, or to compare 100 JSON objects with title and author for correctness?
The example is only partly serious. I just think that as long as hallucinations occur, Generative AI will only get part of my trust - and I don't know about you, but if I knew that a person was outright lying to me in 3% of all his statements, I wouldn't necessarily seek his proximity in things that are important to me...