"B...b...b...but it's just autocomplete." -- HN
With the playground's defaults of t=0.8, p=0.9, I got the right answer 7/10 times.
With t=0.1, p=0.1, it's always exactly "The name of the cat that wears a blue tie is Jackson."
Prompt: When I was born, my biological mother was in Paris, and my father was in the United States. Where was I born?
GPT-4: Based on the information you've provided, I can't definitively say where you were born. Your location of birth depends on various factors, such as where your mother was living around the time of your birth, or if she was traveling. If you were born when your mother was in Paris, then you were likely born in Paris. However, if she traveled to the United States or another location for your birth, you would have been born there. Additional information would be needed to determine your birth location more accurately.
Falcon40B: You were born in Paris. Your biological mother was in Paris when she gave birth to you.
This kind of evaluation is meaningless, it would be necessary to test the models on a bunch of different prompts to get some meaningful metrics.
Falcon 40B:
> I've got two cats named Buttons and Jackson. They both like to wear cute little ties. Buttons wears a red tie. My other cat wears a blue tie. What is the name of the cat that wears a blue tie?
> I'm sorry, but I do not have enough information to answer your question. Could you please provide me with more context or information?
ChatGPT:
> I've got two cats named Buttons and Jackson. They both like to wear cute little ties. Buttons wears a red tie. My other cat wears a blue tie. What is the name of the cat that wears a blue tie?
> The cat that wears a blue tie is named Jackson.