Re:Reflection 70B, the top open-source model
by winddude on Thursday September 05, 2024 @05:13PM
(#41460454)
PhDs aren't relevant. It's more just a certificate that you can learn to learn and stay committed to hard and challenging things. It does give bonus points to VCs, because it's seems to be easier to market to other VCs, same applies for hedge funds.
And with fine tuning, there's zero math needed, it's a bit of common sense, and a lot's of data optimization.
Re:Reflection 70B, the top open-source model
by phs318u on Thursday September 05, 2024 @10:17PM
(#41462269)
I wouldn't say that PhD's aren't relevant. Remember a lot of this subsequent "bumps, steps and leaps" advancement has come _after_ the initial work by the OpenAI's etc. "Standing on the shoulders of giants" is a thing.
Re:Reflection 70B, the top open-source model
by sabbaticaldev on Friday September 06, 2024 @11:49PM
(#41471318)
and these phds used some tools developed by teenager hackers. Standing on the shoulders of giants, indeed
Re:Reflection 70B, the top open-source model
by yaj54 on Thursday September 05, 2024 @05:25PM
(#41460533)
Anyone have or know of a list of LLM challenges like this? Targeted use cases with unpublished test data?
Re:Reflection 70B, the top open-source model
by moralestapia on Thursday September 05, 2024 @04:57PM
(#41460355)
>A friend who developed NoCha [1] is running the model on that benchmark [...]
Please do update us on the result.
Re:Reflection 70B, the top open-source model
by nsagent on Friday September 06, 2024 @04:11PM
(#41469353)
Not looking good. Apparently the model was broken when they released it yesterday. The version they uploaded 8hrs ago only has an 8k context length, so we can't test it on the novels.
Here's the updates to the model config on huggingface:
https://huggingface.co/mattshumer/Reflection-Llama-3.1-70B/c...
Re:Reflection 70B, the top open-source model
by m3kw9 on Thursday September 05, 2024 @06:33PM
(#41460952)
Fine tuning needs $$$ and knowledge on how fine tuning works.
That said, I'm withholding judgment on how likely the claims are. A friend who developed NoCha [1] is running the model on that benchmark, which will really stress test its ability to reason over full novels. I'll reserve judgement until then.
[1]: https://novelchallenge.github.io/