There are plenty of examples of encoders, like this paper, that are being applied to read through all literature, but using an LLM directly (typically referring to the decoder) is not really necessarily useful or efficient (for typical tasks done in the field). It's far more efficient to use smaller models for these tasks, because, for example, if you have 60 million articles with ~2000 tokens, and your system can process only 30 tokens per second, you're looking at 16 *years* to process the whole dataset. 1*10^1 tokens/second is the order of magnitude that LLMs work in, and maybe 10^2 regime if you have a machine you purchase for around $1 million. So that's years of processing for any typical researcher trying to do this, or months of you have a small amount (single millions) of investment. If it's billions, more possibilities open up as is clear with Deepmind et al.

So clearly you need to have some very good hardware to process all of it.

However, compare that with some of the simpler encoder models that have far fewer params that are targeted for specific tasks. These systems can plow through 10^5 or 10^6 tokens per second. So now that 16 years is a week.

This is why small models that are task specific are so important. They make much more possible in reasonable time frames and reduce CO2 emissions by orders of magnitude. Along the same lines of "why use an LLM to extract everywhere the string 'Starbucks4{:digit:}' appears in text when you can use a regex?". You can get the output in a few seconds on billions of articles with a DB, whereas an LLM would take more than a decade.

Regex? Just use Langchain agents, bro!

:-P

I've also got remarkably good results pushing regular expressions to the max to perform "NER", while other more fancy (ML) solutions failed. So don't rule them out.

Perhaps... but If that's true, you're likely not doing what is typically referred to as NER in computational linguistics. Either that or yours going with a type of 'dictionary based' regex system, which gets typically high recall, low precision. Several years ago, when manual FE was a thing, this would be fed to a CRF model or some other Markov like model, perhaps even an LSTM prior to W2V, and be used as a feature - along with other things like 'ends in ly' etc.

There's always tradeoffs. Regex is fast, linear CRFs are quite fast too. Simple LSTMs are fast-ish, as well as BERT-like systems, can offer a decent tradeoff in speed/performance. LLMs are much slower, and need some type of distill step by step to get anything very useful out (for task specific model).

Ultimately, you're right in that regexes along with other older techniques should be understood and weighed for what is the optimal solution for the task.

I would think the problem is that these LLMs don’t have a concept of any reality/objects outside of language. So many of the insights that come intuitively to humans, like Newton’s apple, will probably not come to an LLM. But I’m really just guessing. I also wouldn’t have thought that ChatGPT can draw a unicorn, and yet apparently it can.

Google have developed Med-Palm 2 for this purpose, https://cloud.google.com/blog/topics/healthcare-life-science...