Re:Intel Neural-Chat-7B Model Achieves Top Ranking on LLM Le...
by colinmorelli on Sunday December 03, 2023 @10:28AM
(#38507732)
I'd say they all "got it wrong" insofar as it's clear they don't seem to grasp the idea that the two things are the same, but are instead providing a response _as if_ one were comparing feathers to some other material.
Or, rather, perhaps it would make more sense to say they "got it different" than a human probably would, who might clarify that you intended to compare a thing to itself or simply say "They're identical, so they weigh the same" or something of that nature.
Instead, the models all explain volume, density, etc which are applicable if you're comparing feathers to lead or steel (as is common in their training data, I'm sure), but doesn't make as much sense in this specific question. Yes, yes, I realize different bundles of 1kg of feathers could have different volume and density. But it seems clear that's not the intent of what the models are communicating.
GPT-4-Turbo:
> A kilogram of feathers and a kilogram of steel weigh the same, as they both are one kilogram. The difference lies in their volume and density, not in their weight.
GPT-3.5-Turbo:
> A kilogram of feathers and a kilogram of feathers both weigh the same, as they both have a mass of one kilogram. The difference lies in the volume and density, with feathers being much less dense than most other materials, so you would need a much larger volume of feathers to make up a kilogram compared to a denser material like metal.
OpenChat Aura
> A kilogram of feathers and a kilogram of feathers both weigh the same, which is 1 kilogram. Therefore, the answer is both weigh the same.