Not my experience at all. 25% sounds really high. I can't even remember the last time it gave me an error that wasn't reasonable (e.g. based on incomplete information) and was just pure noise.

Im curious what types of tasks you're using it for?

fwiw, i dont mean to suggest AI is pure noise or even that AI isnt worth using. the report just doesnt reconcile with my experiences at all.

my experiences range from helping design penn's new AI degree programs, hearing from friends at algorithmic hedge funds, hearing from friends at startups, and my own development.