For coding Claude Opus-3 provides far more mature code and good at finding bugs (when present with the error code) compared to GPT-4-Turbo and GPT-4o. Last few days I've been using both for some python+pyspark project. Not sure how come in their comparison GPT-4o is showing that good!

100% agree here. Claude is especially good at larger context sizes and retains coherence way longer than GPT-4 series of models

I find that it varies between language and task whether GPT-4o or Claude3 Opus will be better. I usually try both now.

I agree. There are some corner cases that GPT-4o reliably fails that Claude does well in, and vice versa. GPT-4 and GPT-4o consistently generates very poor cv2 Python code for human face/boundary box work - it's a strange reproducible failure in my experience.

GPT4(o) is quite good at advanced math, it's been helpful when I was learning differential geometry. Not sure how Claude compares though, this 3.5 release has tempted me to try it out. Also, it's finally available in Canada!

Claude 3 was much better than GPT4 for functional analysis and abstract algebra (first year classes).

One huge leg up here is ChatGPT defaults to outputting (and actually displaying, if you're using the default client) LaTeX. Between that and this being one of the few places high verbosity is actually helpful I preferred GPT4/4o for helping learn calc 2. It's well possible Claude 3.5 Sonnet gets the final answer right on the first try more often though.

After a few months of writing all my homework in LaTeX I'm finding my thinking slid towards the raw latex rather than the rendered form. I'll have to wait till fall semester to give 3.5 a good whirl.

I'm honestly shocked people are saying this. I use both and GPT-4 is usually better.

What kind of coding tasks is Claude 3 opus doing for people ?

I'm surprised there isn't a single mention of Gemini 1.5 Pro. I've been using it for about a month because it came for free with my Google setup and I've been pretty happy. Not for coding but mostly for business tasks like writing minutes from transcripts, summarizing long legal documents,... and the long context length has been awesome. It also conveniently integrates with the rest of my google setup like Drive.

IIRC it also ranked only behind gpt4o on benchmarks.

Gemini in general is terrible. Way too many mistakes. If you use it via the API it repeats itself constantly. At least it's the model that is the easiest to jailbreak and will happiliy give you a tutorial on how to make a bomb if you ask politely ;) Very ironic considering how Google emphasizes "safety".

I've also had good results with Gemini 1.5 Pro for some tasks. Just yesterday, it produced very good analysis and comments based on a 200-page document. ChatGPT 4o was much weaker, and the document was too large for Claude 3 Opus. (This was a few hours before 3.5 was released.)