News for hackers. Stuff that matters.

A comment on g1: Using Llama-3.1 70B on Groq to create o1-like reasoning chains. Parent comment

Re:g1: Using Llama-3.1 70B on Groq to create o1-like reasoni... by whimsicalism on Monday September 16, 2024 @03:36PM (#41559750)
> believes Strawberry is mainly just CoT. I'm not saying they didn't fine tune a model too

You don't see the scaling with respect to token length with non-FT'd CoT like this, in my opinion.

[ Parent | on HN ]
Re:g1: Using Llama-3.1 70B on Groq to create o1-like reasoni... by quantadev on Monday September 16, 2024 @04:03PM (#41560102)
I haven't even added Strawberry support to my app yet, and so haven't checked what it's context length is, but you're right that additional context length is a scaling factor that's totally independent of whether CoT is used or not.

I'm just saying whatever hey did in their model, I think they also added CoT on top of it all, as the outer layer of the onion so to speak.

[ Parent | on HN ]