In your analogy that calculator would only produce a correct answer 80% of the time, and plausible looking but incorrect ones the other 20%.

If that were the case I’d hire pen guy.

What's the error rate of the pen guy?

Also, if your AI has a 20% error rate, you're not holding it right. You need to spend more time keeping it on rails - unit tests, integration tests, e2e tests, local dev + browser use, preview deployments, staging environments, phased rollouts, AI PR reviews, rolling releases. The error rate will be much closer to 0%.

How does a phased rollout improve LLM error rates exactly?

Error rate here is the rate of shipping bugs to customers.

That wasn't what the comment you responded to was referring to. I guess it makes sense since you are kind of like an LLM with how you respond to input.

More like “Producing 80% of the correct answer” and the remaining 20% with some nudging and tweaking. Still extremely valuable.

> I feel the same way about the current crop of AI tools. I've tried a bunch of them. Some are good. Most are a bit shit. Few are useful to me as they are now. [...] If this tech is as amazing as you say it is, I'll be able to pick it up and become productive on a timescale of my choosing not yours.

I think the point the author is making is not that it's all useless, but against the very overly simplistic idea the plot of Amount of AI vs Productivity in All Situations is a hockey stick chart.

Being told to be excited about something when clearly all they're saying is "it works sometimes, other times not so much. I'll keep checking and when it's good enough for me I'll get on board" is aggravating.

To be honest, I would rather spend 5x effort while doing my normal work, because salary won't grow.

> Instead of doing 1x your normal work, you can do 5x while still maintaining quality.

That's a gross overestimate. 2x I would maybe believe.

Someone has to sign off your work and unless it's hard to write but easy to read, this is where the bottleneck currently lies.

Everything is relative. If your systems aren't adapted to AI development, it will be much lower.

> Instead of doing 1x your normal work, you can do 5x while still maintaining quality.

Yet my pay stays the same, all my coworkers get fired, and Sam Altman gets all of their paychecks. Hrm.

What if the calculator had randomness built into it?