News for hackers. Stuff that matters.
Re:Advanced NLP with SpaCy by roenxi on Saturday September 09, 2023 @04:24AM (#37443313)
1) It is extremely rare for a field to ever 'be solved'. There is still active research into how to multiply 2 numbers together. NLP is not anywhere close to solved.

2) LLMs have different trade-offs to fundamental techniques. Linear regression still gets lots of use despite there usually being a theoretically better method for any specific application. There will be parallels to that in NLP.

3) Isn't the article talking about training things like LLM? It is right there - "Chapter 4: Training a neural network model".

Re:Advanced NLP with SpaCy by visarga on Saturday September 09, 2023 @04:54AM (#37443450)
Funny thing, you can get LLM+Code Interpreter to do your linear regression with sklearn. So LLM+toys "can do" linear regression, or any other library algorithm.

And I think I read in some paper that pure LLMs can do regression tasks in few shot mode. Maybe not as good as sklearn, but a decent result.

Re:Advanced NLP with SpaCy by roenxi on Saturday September 09, 2023 @05:28AM (#37443594)
I'm unimpressed. I'd guess that every modelling technique can do linear regression. It is literally drawing a line through data. If you are lucky enough, even a one parameter model can get pretty close to what a linear regression does. But doing linear regression with LLMs is wasteful. The computational power alone makes it a party trick and not much more.
Re:Advanced NLP with SpaCy by axpy906 on Saturday September 09, 2023 @11:34AM (#37446579)
Large Language Model in my opinion means that it has billions of parameters and possibly cannot fit on a single machine.

Not sure what Spacy is doing under the hood these days but always thought of “neural net” as word2vec type model which won’t hit the above definition.