Re:Don't believe the hype: why ChatGPT is not the “holy grai...
by esjeon on Friday May 12, 2023 @02:06PM
(#35919809)
I think the analogy doesn't work here.
In my understanding, LLMs already hit a big wall. We can't increase the size of models mainly because it's too expensive, but also doing so may not be as effective as before. We've also run out of data. The free lunch is likely already over, for now. It's unlikely that we'll see huge improvements in the direction we've seen during recent years.
Instead, what I see is that the first letter 'L' is getting smaller. People are working on (relatively) smaller specialized models. But it means these models are unlikely outperform larger LLMs (in the direction mentioned above).
In my understanding, LLMs already hit a big wall. We can't increase the size of models mainly because it's too expensive, but also doing so may not be as effective as before. We've also run out of data. The free lunch is likely already over, for now. It's unlikely that we'll see huge improvements in the direction we've seen during recent years.
Instead, what I see is that the first letter 'L' is getting smaller. People are working on (relatively) smaller specialized models. But it means these models are unlikely outperform larger LLMs (in the direction mentioned above).