> but that overwhelmingly doesn’t work.
MCTS will be the next big “thing”; not agents.
They are not mutually exclusive. Likely we'll get more clear separation of architecture and underlying technology. In this case agents (i.e. architecture) can use different technologies or mix of them. Including 'AI' and algorithms. The trick is to make them work together.
Actually o1, o3 are doing exactly this, and very well. I.e. explicitly: by proper orchestration the same LLM can do much better job. There is a price, but...
> you would expect to be able to build agentic systems out of much smaller LLMs
Good point, it should be possible to do it on a high-end pc or even embedded.