Re:LLMs will fail this simple eval every time
by ifuknowuknow on Saturday June 15, 2024 @08:30PM
(#40693727)
1) can't tell any one specific thing it doesn't know
2) when it does tell a specific thing it doesn't know, it's a thing it actually knows well
this train of conversation looks monkeypatched by openai bc it reveals a massive vulnerability in reasoning, but is consistently reproducible across all llms
2) when it does tell a specific thing it doesn't know, it's a thing it actually knows well
this train of conversation looks monkeypatched by openai bc it reveals a massive vulnerability in reasoning, but is consistently reproducible across all llms