The 70% problem: Hard truths about AI-assisted coding

Re:The 70% problem: Hard truths about AI-assisted coding by cvwright on Friday December 06, 2024 @10:11AM (#42340498)

I think of AI like a very eager and bright 20 yr old intern who’s really into whatever tech I’m trying to learn.

Good knowledge of facts, willing to give anything a try, but not much wisdom.

[ Parent | on HN ]

Re:The 70% problem: Hard truths about AI-assisted coding by dartos on Friday December 06, 2024 @10:28AM (#42340670)

> Good knowledge of facts

I haven’t seen this demonstrated in gpt-4 or Claude sonnet when asking anything beyond the most extreme basics.

I consistently get subtly wrong answers and whenever I ask “oh okay, so it works like this” I always get “Yes! Exactly. You show a deep understanding of…” even though I was wrong based on the wrong info from the LLM.

Useless for knowledge work beyond RAG, it seems.

Search engines that I need to double check are worse than documentation. It’s why so many of us moved beyond stack overflow. Documentation has gotten so good.

[ Parent | on HN ]

Re:The 70% problem: Hard truths about AI-assisted coding by tmpz22 on Friday December 06, 2024 @11:49AM (#42341427)

ChatGPT and Claude outputs are equivalent to a 20 yr old intern who goes to MIT?

[ Parent | on HN ]