Prompt: What’s heavier, a kilogram of steel or two kilograms of one kilogram feathers?
GPT-4: Two kilograms of one-kilogram feathers are heavier than a kilogram of steel. Despite the misconception caused by the popular question about what's heavier—a kilogram of steel or a kilogram of feathers (they are equal)—in this case, you are comparing two kilograms of feathers to one kilogram of steel. Hence, the feathers weigh more.
Prompt: What’s heavier, a kilogram of steel or two kilograms of one kilogram feathers?
GPT4: Two kilograms of feathers are heavier than one kilogram of steel. The weight of an object is determined by its mass, and two kilograms is greater than one kilogram, regardless of the material in question.
"Right, but they're both a kilogram"
From the answers you (and the others) have obtained, however, I’m not convinced that OpenAI aren’t just “hardcoding” fixes to the traps that become popular. Sure seems like it still can’t logic it’s way around weight.
They are, you just have to be creative with it. And what they demonstrate is that all of these LLM's can't reason, they only know how to parrot back what they think you want.
"What’s heavier, a kilogram of steel or two kilograms of one kilogram feathers?"
GPT: A kilogram of steel is heavier than two kilograms of feathers.
"Why is a kilogram of steel heavier than two kilograms of feathers?"
GPT: This is because steel is a much denser material than feathers. Steel is made up of atoms that are much closer together than the atoms that make up feathers, making it heavier for its size.