Isn't o1 just applying last year's Tree of Thoughts paper in production? Is there any reason to believe that the other companies will struggle to implement their own?

https://github.com/princeton-nlp/tree-of-thought-llm

I don't think it's tree of thoughts at all.

I think it's as they say: reinforcement learning applied to cause it to generate a relatively long 'reasoning trace' of some kind from which the answer is obtained through summarisation.

I think it's likely a cleverly simplified version of QuietSTaR, with no thought tokens, just one big generation to which the RL is applied.

The way I believe it's trained in practice is as follows: they have a bunch of examples, some at the edge of GPT-4s ability to answer, some beyond it, some that GPT-4 can answer if you're lucky with the randomness. Then they give it one of these prompts, generate a fairly long text, maybe 3x the length of the answer, and summarize that to produce the final answer. Then they use REINFORCE to reward the generated texts that increase the probability of the summary being correct.

Not be be nitpicky, but being the first to deploy recent academic research papers to production should count as a breakthrough IMHO.

There seems to be several components involved: tree of thoughts, MCTS, RL, high quality chain of thought data, possibly multiple models.

CoT can be _easily_ achieved using langgraph in a similar manner. There’s no “scaling of inference” it’s just prompting, all the way down.

It's not just CoT

o1 seems like it’s basically 4o with some chain of thought bolted on. Personally, I don’t consider chain of thought a breakthrough, let alone a major one.

It's much more than CoT. I suspect it will take other labs some time to replicate.

> o1 is a major breakthrough

Is it? I feel like if you don't care about the cost it's pretty easily replicable on any other LLM, just with a lang-chain sort of approach

Then why hasn't it been done?

Someone gave the various models an IQ test and the previous ones scored 80-90 so a bit dim compared to humans, and o1 got 120, quite bright for a human. https://www.reddit.com/r/ClaudeAI/comments/1fhwfyl/openai_1o...

this may have consequences for how useful it is.