Hi, I am curious, what do you think are the parts that are obvious but still work in progress? Maybe continuous multimodal input?
I'm not fully up to date on AI research anymore, however I think the obvious stuff has already been implemented, but there's still a lot of room for improvement and scaling for these current approaches.
For verifiable tasks like coding and math, you can keep going, doing more and more RL. You have the AI generate hard problems, then try to solve them and reward correct solutions. We basically have RSI for verifiable tasks, we just need to keep the wheel turning.
Even a nobody like me was playing with a sort of CoT with llama finetunes before o1 came out.
Only this year I'd say we picked all the low hanging fruit that has been obvious for a while that was needed.