“Is your question right? Did you intend to have kilograms of feathers in there twice?” this is how assistants should answer, or just directly say they’re the same.

Here’s how GPT4 answered it for me

“A kilogram of feathers and a kilogram of steel weigh the same. Both are one kilogram. The difference lies in their volume and density, not their weight. Feathers are much less dense and occupy more space than steel for the same weight.”

And it does if you start it off with a prompt that asks it to check to make sure your inputs are valid.

Yes. That’s the point, instead of assuming I asked the unrelated trick question it should say they’re the same and not answer about random dense materials that are typically used.

As a human, I’m not even sure what the correct answer should be.

The pedantically correct response is to point out that you’re asking which of two identical things is different. As some commenters here point out, error correcting assumptions are a feature of intelligence. Geeks are sometimes mocked for taking words too literally.

But the questions are related. It's following a pattern that the AI has seen before and will respond in that spirit. Of course, humans are also prone to projecting the expected pattern in the face of a slight deviation. So I don't see how this is a knock on LLMs.

Maybe the model could think that you're asking the trick question wrongly like the parent comment was wondering.

To be honest I didn't see you change the question until I saw the model response.

This would be an acceptable response.