I think it depends on how the question is constructed: - I want to do X, how do I do it? - I was thinking of doing X to achieve Y, wonder if that's a good idea?

Sometimes, I really want to do X, I know it may be questionable, I know the safest is "probably don't want to do it", and yet, that's not someone else's (or LLMs) business, I know exactly what I want to do, and I'm asking if anyone knows HOW, not IF.

So IMO it's not a flaw, it's a very useful feature, and I really do hope LLMs stay that way.

I think there can be a middle ground. I think it's fine if LLMs warn you but still answer the question the way you asked. I don't always know when i should be asking if something is a good idea or not.

I always wonder about that. Very often it seems that you need to be able to LLM that they are wrong. And then they happily correct themselves. But if you do not know that the answer is wrong how can you get correct answer?

Worse: if you think the LLM is wrong and try to correct it, it will happily invent something completely different (and actually wrong this time).

This happened to me the other day. I had framed a question in the ordinal case, and since I was trying to offload thinking anyways, I forgot that my use case was rotated, and failed to apply the rotation when testing the LLM answer. I corrected it twice before it wrapped around to the same (correct) previous answer, and that’s when I noticed my error. I apologized, added the rotation piece to my question, and it happily gave me a verifiably correct answer.

Sure but most common LLMs aren't going to be patronizing and presumptuous while they say so