LLMs will fail this simple eval every time

Re:LLMs will fail this simple eval every time by mtmail on Saturday June 15, 2024 @06:16PM (#40693102)

Can you explain the failure?

[ Parent | on HN ]

Re:LLMs will fail this simple eval every time by ifuknowuknow on Saturday June 15, 2024 @08:30PM (#40693727)

1) can't tell any one specific thing it doesn't know

2) when it does tell a specific thing it doesn't know, it's a thing it actually knows well

this train of conversation looks monkeypatched by openai bc it reveals a massive vulnerability in reasoning, but is consistently reproducible across all llms

[ Parent | on HN ]