Reflection 70B, the top open-source model

Re:Reflection 70B, the top open-source model by xianshou on Thursday September 05, 2024 @05:06PM (#41460404)

Crazy how simple the technique is if this holds up. Just <think> and <reflection> plus synthetic data, used to finetune Llama 3.1 70B.

Note that there's a threshold for how smart the model has to be to take advantage of this flow (https://x.com/mattshumer_/status/1831775436420083753) - 8B is too dumb.

In which case, what happens if you apply this to a GPT-4o finetune, or to Claude 3.5 Sonnet?

What happens if you combine it with variants of tree-based reasoning? With AlphaProof (https://www.nature.com/articles/s41586-023-06747-5#Sec3)? With MCTSr (https://arxiv.org/abs/2406.07394)?

[ Parent | on HN ]

Re:Reflection 70B, the top open-source model by jug on Thursday September 05, 2024 @05:29PM (#41460557)

I was just thinking - since GPT-4o and Sonnet are closed models, do we know that this method was not already used to train them? And that Reflection is simply finding a path for greater improvements than they did. Llama 3.1 apparently didn't improve as much. It's just a thought though.

[ Parent | on HN ]

Re:Reflection 70B, the top open-source model by hdeezy on Thursday September 05, 2024 @09:42PM (#41462093)

If they had, this thing wouldn't be trading punches with them at its size

[ Parent | on HN ]

Re:Reflection 70B, the top open-source model by segmondy on Thursday September 05, 2024 @11:53PM (#41462736)

Sonnet does something like this. See - https://tyingshoelaces.com/blog/forensic-analysis-sonnet-pro...

[ Parent | on HN ]

Re:Reflection 70B, the top open-source model by itorcs on Friday September 06, 2024 @12:01PM (#41467321)

What parameter size are 4o and sonnet?

[ Parent | on HN ]