News for hackers. Stuff that matters.
Re:Falcon 180B Released by ttul on Wednesday September 06, 2023 @01:29PM (#37408215)
Falcon fails. GPT-3.5 also fails this test. GPT-4 gets it right. I suspect that GPT-4 is just large enough to have developed a concept of counting, whereas the others are not. Alternatively, it's possible that GPT-4 has memorized the answer from its more extensive training set.
Re:Falcon 180B Released by mk67 on Thursday September 07, 2023 @10:23AM (#37419313)
It's not possible to count letters for an LLM; it only "sees" tokens.