Ouch. Was that necessary?
I used $1000 worth of GPU credits and threw in VADER because it’s basically free both in time and credits.
I usually do this on large dataset out of pure interest in how it correlates with expensive methods on English language text.
I am well aware of how VADER works and its limitations, I am also aware of the limitations of all sentiment analysis.
I can't really think of many NLP things that are one-decade old and don't have a better / faster / cheaper alternative.
It's like saying I chose barefoot walking to get to the next town and while admittedly it was a painfull and not pleasant experience, it was free. I did try a helicopter service but that was very expensive for my use case.
People are pointing out you could have used a bicycle instead.
I mean he did use the state of the art for his work, he just checked how much better it actually was in comparison to a much simpler algorithm and thought the cost/benefit ratio to be questionable... At least that's what I read from his comments
That seems to me like it would flat out fail on sarcasm. How is that still considered a usable method today?
b) compute costs are plummeting. inference in the cloud costs has dropped over 80% in 1 year
c) similar to a), spending a little more and having a beefy enough machine is functionally cheaper after just a few projects
d) everyone trying to do sentiment analysis is trying to make waaaay more money anyway
so I dont see NLP’s even lower costs of being that relevant. its like pointing out that I could use assembly instead of 10 layers of abstraction. It doesnt really matter
I’ve also found them surprisingly difficult and non-intuitive to train, eg deliberately including bad data and potentially a few false positives has resulted in notable success rate improvements.
Do you consider BERTs to be the upper end of traditional - or, dunno, transformer architecture in general to be a duff? Am sure you have fascinating insight on this!
I think it would be safe to say BERT is/was firmly in the non-traditional side of NLP.
A variety of task specific RNN models preceded BERT, and RNN as a concept has been around for quite a long time, with the LSTM being more modern.
Maybe word2vec ushered in the end of traditional NLP and was simultaneously also the beginning of non-traditional NLP? Much like Newton has been said to be both the first scientist and also the last magician.
I find discussing these kind of questions with NLP academics to be awkward.
it's like:
"does apple cure cancer in monkeys?" vs "does blueberry cure diabetes in pigs?"
I try to get the best of both words by using LLMs to generate synthetic data to train NLP classifiers. First, I use LLMs to generate variations of human-labeled data. Second, I use LLMs to label unlabeled data.
In a future challenge, I want to train LLMs to generate data to train NER for segmenting documents and extracting information.
I can confirm LLMs have essentially confined a good chunk of historical research into the bin. I suspect there are probably still a few PhD students working on traditional methods knowing full well a layman can do better using the mobile ChatGPT app.
That said traditional NLP has its uses.
Using the VADER model for sentiment analysis while flawed is vastly cheaper than LLMs to get a general idea. Traditional NLP is suitable for many tasks people are now spending a lot of money asking GPT to do just because they know GPT.
I recently did an analysis on a large corpus and VADER was essentially free while the cloud costs to run a Llama based sentiment model was about $1000. I ran both because VADER costs nothing but minimal CPU time.
NLP can be wrong but it can’t be jailbroken and it won’t make stuff up.