The Prospect of an AI Winter

Re:The Prospect of an AI Winter by codelord on Monday March 27, 2023 @06:00PM (#35333334)

IMHO the prospect of an AI winter is 0%. As someone who has done research in ML I think ML technology is moving forward much faster than we anticipated. ChatGPT shouldn’t work based on what we knew. It’s incredible that it works. Makes you think what other things that shouldn’t work we should scale up and see if they would work. And then there are things that we think they should work. Each new paper or result opens the door for many more ideas. And there are massive opportunities in applying what we already have to all industries.

You can absolutely build high precision ML models. Using a transformer LM to sum numbers is dumb because the model makes little assumptions about the data by design, you can modify the architecture to optimize for this type of number manipulation or you can change the problem to generating code for summing values. In fact Google is using RL to optimize matmul implementations. That’s the right way of doing it.

[ Parent | on HN ]

Re:The Prospect of an AI Winter by dragonwriter on Monday March 27, 2023 @06:05PM (#35333396)

> As someone who has done research in ML I think ML technology is moving forward much faster than we anticipated.

The past AI winters have been preceded by periods of AI moving forward faster than anticipated. Its when the limits of easy advancement with the current approaches are reached without a new approach that allows continued rapid progress you get an AI winter.

[ Parent | on HN ]

Re:The Prospect of an AI Winter by RivieraKid on Monday March 27, 2023 @07:02PM (#35334041)

> ChatGPT shouldn’t work based on what we knew

Why?

[ Parent | on HN ]

Re:The Prospect of an AI Winter by layer8 on Monday March 27, 2023 @11:05PM (#35335371)

I guess because we still don’t understand why it works. We still have no basis on which to predict that it works as well as it does, except for the manifest fact that it does.

[ Parent | on HN ]