g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains

Re:g1: Using Llama-3.1 70B on Groq to create o1-like reasoni... by kristianp on Sunday September 15, 2024 @08:49PM (#41551741)

Does o1 need some method to allow it to generate lengthy chains of thought, or does it just do it normally after being trained to do so?

If so, I imagine o1 clones could just be fine tunes of llamas initially.

[ Parent | on HN ]

Re:g1: Using Llama-3.1 70B on Groq to create o1-like reasoni... by astrange on Monday September 16, 2024 @04:53AM (#41554090)

You need an extremely large amount of training data of good CoTs. And there probably is some magic; we know LLMs aren't capable of self reflection and none of the other ones are any good at iterating to a better answer.

Example prompt for that: "give me three sentences that end in 'is'."

[ Parent | on HN ]