Intel Neural-Chat-7B Model Achieves Top Ranking on LLM Leaderboard

Re:Intel Neural-Chat-7B Model Achieves Top Ranking on LLM Le... by digbybk on Sunday December 03, 2023 @10:29AM (#38507736)

I wouldn’t even say it’s failing. Recognizing and correcting obvious errors in speech is a feature not a bug, both in humans and LLMs. Is it more likely that they meant to say feather twice or they mistyped?

[ Parent | on HN ]

Re:Intel Neural-Chat-7B Model Achieves Top Ranking on LLM Le... by transcriptase on Sunday December 03, 2023 @10:32AM (#38507766)

True, but would it not make more sense to ask rather than assume and provide a long winded explanation?

And even if it was a typo and I meant to say rocks, the model will often assume a variety of metals and answer with those.

[ Parent | on HN ]

Re:Intel Neural-Chat-7B Model Achieves Top Ranking on LLM Le... by qup on Sunday December 03, 2023 @10:36AM (#38507793)

How it proceeds in the case of human error is going to be almost totally removed from its "intelligence," and almost totally related to its training.

[ Parent | on HN ]

Re:Intel Neural-Chat-7B Model Achieves Top Ranking on LLM Le... by digbybk on Sunday December 03, 2023 @10:40AM (#38507839)

I’m not claiming otherwise here. I’m claiming that it’s not a good test because a human would (almost certainly correctly) also skip past the double feather and answer the likely spirit of the question.

[ Parent | on HN ]

Re:Intel Neural-Chat-7B Model Achieves Top Ranking on LLM Le... by throwuwu on Sunday December 03, 2023 @10:38AM (#38507820)

A human would probably say “did you mean a kg of feathers and a kg of lead?”

[ Parent | on HN ]