Claude 3 beats GPT-4 on Aider's code editing benchmark

Re:Claude 3 beats GPT-4 on Aider's code editing benchmark by aubanel on Thursday March 28, 2024 @02:01PM (#39855200)

Even in the most restrictive définition of recall as in "retrieve a short contiguous piece of information inside an unrelated context", it's not that good. It's always the exact same needle inserted in the exact same context. Not the slightest variation apart from the location of the needle.

Then if you want to test for recall of sparse information or multi-hop information, it's useless.

[ Parent | on HN ]

Re:Claude 3 beats GPT-4 on Aider's code editing benchmark by BA2255 on Thursday April 11, 2024 @02:25AM (#39998966)

For my education, how do you use the 200k contenxt the normal chats like Poe, or chatgpt don't accept longer than 4k maximum. Do you use them in specific Playgrounds or other places?

[ Parent | on HN ]

Re:Claude 3 beats GPT-4 on Aider's code editing benchmark by aubanel on Monday April 15, 2024 @03:13AM (#40037767)

The calls with long context are done through specific APIs, that you can call for instance in Python or Javascript.

Here's a quick start guide with OpenAI: https://platform.openai.com/docs/quickstart?context=python

[ Parent | on HN ]