News for hackers. Stuff that matters.

A comment on Qwen2 LLM Released. Parent comment

Re:Qwen2 LLM Released by GaggiX on Thursday June 06, 2024 @01:03PM (#40599772)
>that also makes good use of that full window?!

To be honest the "Needle In A Haystack" test is the most trivial test for a model that relies on full attention, it's expected to be easy to pass if the model was trained correctly.

[ Parent | on HN ]
Re:Qwen2 LLM Released by ai_what on Thursday June 06, 2024 @01:16PM (#40599945)
I agree. I personally don't have high hopes for the 0.5B model.

Phi-2 was 2.7B and it was already regularly outputting complete nonsense.

I ran the 0.5B model of the previous Qwen version (1.5) and it reminded me of one of those lorum ipsum word generators.

The other new Qwen models (7B and up) look good though.

[ Parent | on HN ]
Re:Qwen2 LLM Released by refulgentis on Thursday June 06, 2024 @02:24PM (#40600699)
Phi-2 wasn't instruct/chat finetuned and it was very upfront about this, "I tried Phi-2 and it was bad" is a dilletante filter
[ Parent | on HN ]
Re:Qwen2 LLM Released by falseAss on Saturday June 08, 2024 @06:17PM (#40620554)
i just hope people don't claim "X model support Y context window", when the evaluation is done on "Needle in a haystack" only. It creates so much unnecessary hype.
[ Parent | on HN ]