Agreed on ML being in production in a lot of places and has been quite valuable, particularly for large incumbents. I write about this a bit here:
https://blog.eladgil.com/p/ai-startup-vs-incumbent-value
I think the difference this time is the types of capabilities provided by transformers vs prior waves of AI are sufficiently different to allow many more types of startups to emerge, as well as big changes in some types of enterprise software by incumbents - in ways that were not enabled by pre-existing ML approaches.
Re:Early Days of AI
by vasilipupkin on Monday August 21, 2023 @07:20PM
(#37216638)
The jury is still out on how useful these additional capabilities provided by transformers are. The question is what is the degree to which it’s possible to reduce the frequency and severity of hallucinations. If that degree is limited without major changes in architecture or major new breakthroughs, then usefulness of gpt-4 style models will be limited. And we just don’t know the answer yet. So far, usefulness of gpt-4 is real but extremely limited. Another issue is this approach means models are costly to train and don’t easily incorporate latest info about the world. In short, it’s way too early to hype this up.
But let's not confuse that in general with AI or Machine Learning which is already used heavily in lots of places.
the specific type of architecture that gpt-4 uses for next word prediction is not the only possible architecture and is not what's used for many real world tasks. There a lot of different problems being addressed by ML and next word prediction is just one of them, although quite important.