this

Considering this hockey stick is in the field that is in the business of making hockey sticks it could go pretty far. A hockey stick maximizer if you will.

This is the big question isn't it. With self driving cars we have been on the final 20% for what seems like forever.

Big difference here is that it becomes more useful as it gets better. Self driving cars aren't useful until they reach a certain threshold.

I think there is a feedback loop with self-driving cars.

More self-driving cars on the road -> more driving data for that company -> better self driving cars

We are seeing continuous improvements in image and video generation / understanding every year. It was just probably too naive to think that self driving can work well without getting to human level video understanding (which is still not AGI).

But when we have that, it’s hard to believe that self driving won’t work.

To me the key issue are these 'hallucinations'--mistakes that seem plausible but are completely made up, like API endpoints that would be super useful except for the small problem that they don't exist. GPT4 is better than GPT3 on these but it still produces a lot of them.

The question is whether these are somehow inherent to the LLM approach or whether scaling up and continued improvements can eventually get rid of them.

They are the main barrier at this point between a very useful tool, but one that still needs to have all its output carefully checked by humans when it comes to anything important, and a true autonomous agent that can be given full tasks to do on its own.

It seems pretty clear to me that you could do some more RL to enforce truth-telling/admitting when it does not know - it would just be much more labor intensive compared to the RLHF they have already done because fact checking is difficult.

I'd imagine they've already been doing lots of RL in this direction, which explains the improvements in GPT4, but it's still an issue. Maybe they can eventually eliminate hallucinations completely, but I could also imagine that it will end up being difficult to do that without lessening its creativity across the board. Perhaps making things up is fundamental to how LLMs work and trying to stop it from doing that will kill the magic. I'm not an AI researcher so I really have no idea--just speculating.

I'm not at all trying to downplay the power or significance of LLMs, btw, in case that's why I'm getting downvoted... I'm using copilot/GPT4 every day and they are massive productivity boosters. But currently I see them as tools for producing rough drafts that need to be revised and checked over. If they can't solve hallucinations, LLMs will stay in this lane, which is still incredible, amazing, and useful, but won't necessarily get us to the AI endgame that the hype is predicting.