With the LLM you don't even need a data scientist to train the model, you just need someone who can connect to APIs. And the LLM is fine with adding and removing categories. No retraining.
It makes the initial faces incredibly fast. Then it is slow and expensive to run, so once the system settles, it becomes the target for model optimization, e.g, a dedicated, or semi-dedicated model.
I see coworkers using AI for tasks like "visit an API endpoint, download a file, put it in S3". This is so stupid. Okay if you want to use AI, but use it to write python code to do that, why pay in tokens every time you want to run that task, it makes no sense
That is the reason.
Once you then have a system that is used enough, it starts making sense to look to replace that with a leaner classifier, but LLMs have a place in vast number of places where we'd previously never use a classifier because it was too much effort.
Absolutely! But that doesn't rule out that there are AIbros that will tell you to use an LLM, even if the task could be done with a static analyzer with a few rules. Use an LLM because it's AI!
Well not anymore... You can just get AI to build it.
Just like, LLMs can build web back ends, but I'd want a security professional to give it some kind of thumbs up. Especially if a lot of money rides on it.
So between llm and pseudo experts, at this stage, I prefer to go with a LLM.
An issue in industry was that companies would not allow use for training on their data across customers. Understandably so, as SVM dimensions are readily interpretable given some thought.
So you'd start from scratch every time and often didn't have enough training data overall.
LLMs "solved" that by just ignoring the previous rules and training on everything, which lets the needed world knowledge seep back in.
I'm sure we'll see many combinations of LLMs and regular machine intelligence in the near future.
You can also do stuff like setup active learning but have the labeller be an llm as well to make things more efficient.
Definitely worth it for PoC level things if not full ones.
If you want to make a ticket classification model, you don't need to train across many customers doing ticket classification, who might not want you to train on their data and whose data is impossible to obtain unless you already are a large company. Instead, you can train on a completely different task which forces the model to learn the syntactic and semantic structure of the English language — say predicting missing words in Wikipedia articles, which you can't do successfully without a good understanding of English — and then fine-tune that model on your specific task with much less data.
Because the initial task is so general, you can have one company do the expensive pretraining, share the base model, and then have everybody else just do the finetuning on their (small) proprietary dataset.
LLMs and models like Jev go one step further, letting you use them as classifiers with no training at all, with nothing but rules in the prompt. And with a few tricks, you can indeed use them to get pretty high-quality labels, which you can then distill into a fine-tuning dataset for a much smaller model.
> say predicting missing words in Wikipedia articles, which you can't do successfully without a good understanding of English
Word2vec is a two layer perceptron.
I feel like people in the field forgot everything.
[1] I went down the rabbit hole of using sequence kernels quite a bit back then, see https://arxiv.org/abs/1910.13800, but the timing was terrible. RNNs and variants like LSTMs were beginning to become popular again, and everyone (including me) switched.
In software engineering, we learned a long time ago that giant monoliths are not the answer. You want smaller, but specialized units that work together. They perform better at their specific task, are more maintainable, you can just upgrade one part of the system without having to impact the others, ....
I don't know where the focus on building a single model that can do everything comes from.
I would assume a smaller model trained specifically to produce code based on natural language input would perform at least just as well, which would already be better given the smaller size / probably lower training resource requirements. This is of course, as already said, an assumption. But generalizing typically comes at a cost, there is no reason why this wouldn't apply to LLMs, it even applies to humans.
The Bitter Lesson is that human-written rule models will always lose out to trained models in the end.
Coding an exercise app will benefit from a model that can code and understands what exercise is. A car showroom website will benefit from a model that understands what relevant filters would be.
Nah, let’s stand firm here and not give in so easily. Parent suggests a coding model needs exercise info in case you want to code an exercise model? Absurd, that leads to the know everything monolith to do anything.
Language, reasoning, tool use, and domain knowledge.. in this case the domain is coding. Someone wants an exercise app? Great, so we look that shit up, we don’t bake it in. We don’t want to bake anything we don’t need in.
We don’t even need to explain this completely reasonable and totally normal engineering preference that small is beautiful. it’s the other side that needs to justify wtf exercise has to do with coding or why everything from twitter posts and a working knowledge of 15th century French poetry might be helpful for serious work in contemporary astrophysics
No, the point is having broader knowledge than just "can code" is important for building software for humans to use.
> We don’t even need to explain this completely reasonable and totally normal engineering preference that small is beautiful.
It is, of course it is, as long as that small thing is useful. My argument is not "let's put literally everything in this thing just in case" it's "broader world knowledge is important for building many things".
Also, we don't look at CVs or histories of people and say "hmm, this person could be good at coding for us but I really wish they didn't have such a broad and expansive history".
> Great, so we look that shit up
But what do you have it look up? What is exercise? What is a muscle and what is a human? How is it going to understand those references? Does it have the broad enough understanding to see where the gaps are?
At some level unless all you want is "loop over the array and sum the results into a variable called sales_total" converted into code, you have to have some broader knowledge about things.
And once you start considering that maybe it needs more than simply code, we're not disagreeing we're just discussing where the line should be. Which I've also not placed, because there's no simple statement to make on that.
Does it? All the latest frontier models that have made technical details publicly available are using MoE; smaller specialized units that work together.
"AGI"
Your model only took minutes to train if you don't count the data collection and labelling work
Saying that the field got reduced to only LLMs is like saying that the field of combustion engines got reduced to cars. The trucks, combine harvesters and peaker plants are still here, they're just in the background, invisible to people who are not part of the industry.
Banks still use random forests and SVMs, medicine and image processing still use CNNs (and people who take photos with their phone don't even realize how much image processing is being done), TTS, ASR and predictive typing models are in wide use, and there's probably still plenty of BERT in spam classification and such.
[1] "Linnaeus: A highly reusable and adaptable ML based log classification pipeline", https://arxiv.org/abs/2103.06927
NOTE: the work was actually accepted at the International Conference on Software Engineering (ICSE) in 2021.