News for hackers. Stuff that matters.
Re:Generative AI is overrated, long live old-school AI by goldenkey on Wednesday March 15, 2023 @01:30PM (#35171812)
When the generative model is autoregressive (autocomplete), it can easily be used as a predictor. All of the state of the art language models are tested against multiple choice exams and other types of prediction tasks. In fact, it's how they are trained...masking - https://www.microsoft.com/en-us/research/blog/mpnet-combines...

For example: "Multiple-choice questions in 57 subjects (professional & academic)" - https://openai.com/research/gpt-4

Re:Generative AI is overrated, long live old-school AI by k8si on Wednesday March 15, 2023 @01:48PM (#35172064)
For GPT4: "Pricing is $0.03 per 1,000 “prompt” tokens (about 750 words) and $0.06 per 1,000 “completion” tokens (again, about 750 words)."

Meanwhile, there are off-shelf models that you can train very efficiently, on relevant data, privately, and you can run these on your own infrastructure.

Yes, GPT4 is probably great at all the benchmark tasks, but models have been great at all the open benchmark tasks for a long time. That's why they have to keep making harder tasks.

Depending on what you actually want to do with LMs, GPT4 might lose to a BERTish model in a cost-benefit analysis--especially given that (in my experience), the hard part of ML is still getting data/QA/infrastructure aligned with whatever it is you want to do with the ML. (At least at larger companies, maybe it's different at startups.)

Re:Generative AI is overrated, long live old-school AI by potatoman22 on Wednesday March 15, 2023 @11:10PM (#35177696)
This is true, but the humans to develop the non-LLM solution are expensive and the OpenAI API is easy.
Re:Generative AI is overrated, long live old-school AI by croes on Wednesday March 15, 2023 @01:52PM (#35172121)
Being good at standardized tests isn't really a good measure.

What happens with completely new questions from totally different subject. The generative model will produce nonsense.