Re:Notes on OpenAI's new o1 chain-of-thought models
by ben_w on Friday September 13, 2024 @01:17AM
(#41528188)
> And how much data can you give it?
128,000 tokens, which is about the same as a decent sized book.
Their other models can also be fine-tuned, which is kinda unbounded but also has scaling issues so presumably "a significant percentage of the training set" before diminishing returns.
128,000 tokens, which is about the same as a decent sized book.
Their other models can also be fine-tuned, which is kinda unbounded but also has scaling issues so presumably "a significant percentage of the training set" before diminishing returns.