News for hackers. Stuff that matters.
Re:AI hype is built on high test scores. Those tests are flawed by Retric on Tuesday October 10, 2023 @09:07AM (#37831575)
Reasoning is the process of using logical inferences to come to some conclusion based on prior knowledge. LLM’s such as GPT 4 don’t do this. Instead they mimic their training data which can work but will also fail in really funny ways sometimes.

GPT 4 has quite a lot of randomness built into its processes so given the same prompt repeatedly it gives somewhat random results. This can make it seem far more accurate than it is when someone keeps trying until they get a correct response. Which is one of the reasons people judge it differently they didn’t necessarily find the correct prompt, frequently the algorithm got lucky.

There are no comments on this comment yet.