I think that makes it a better test. An ideal model would recognize the ambiguity and either tell you what assumption it's making or ask a followup question.
While that is true, I'm not aware of any model that has been trained to do that. And all models can do is to do what they were trained to do.
They are just trained to generate a response that looks right, so they are perfectly capable of asking clarifying questions. You can try "What's the population of Springfield?" for an example.
It's not model but working on top of it: https://www.phind.com/ It's asking clarifying questions.
That's actually pretty good. Thanks!
That's a great point.
You can tease this out pretty easily by having it ask question before continuing. My attempt addresses the ambiguity as the first question:
https://chat.openai.com/share/462a7f62-6305-4e2a-a9ae-5f86a6...
I'll often do this, along with "Are there any mistakes in the above?" or something like "Did this fully address the problem statement?"
Your question actually isn't worded concisely enough. You don't specify whether the string can merely contain the roman numeral (plus other, non-roman-numeral text), or must entirely consist of just the roman numeral. The way "if it contains" is used colloquially, could imply either.
I'd use either "if it IS a roman numeral" if it must consist only of a roman numeral, and "if there exists a roman numeral as part of the string" or some such, otherwise.