Re:LLMs Can't Do Probability
by abdullahkhalids on Wednesday May 01, 2024 @06:24PM
(#40230357)
Isn't this way of prompting roughly equal to asking a 1000 people to pick left or right with 80% prob of left? I imagine, the result with humans will be closer to 80:20 than whatever happened with the LLM.
[ Parent
| on HN ]
I agree it’s equivalent, and that’s a great way to think about it.
But… I wouldn’t be surprised if humans answered closer to the LLM results than 80:20. I’d actually be surprised if humans converged very close to the right result.
Would be a fun mechanical Turk experiment to run.
> Would be a fun mechanical Turk experiment to run.
Sounds like it would be a good way to determine likelihood of MTurk users being LLMs.
Kinda, if those people were clones, with the exact same starting internal state, a d no memory of this ask or ability communicate with each other
Seed a PRNG and it'll return the same result to get random() no matter how many times you execute the test case