For math, I'm kind of surprised that it can't recognize "this is math" and then handle that with normal calculations instead of the language model. I assume we'll see that before long.

A good trick is to ask it to translate the request into commands of your choosing. Like ask it to generate python code to make the calculation for example. Another thing that works well is to turn it into a command extraction problem, give it examples of the kinds of commands you want, and build an interpreter for those commands.

I agree, we’re not far from that, or we’re there now.

I really want Wolfram|Alpha to be integrated into this... that'd be nice. Also if they could make W|A any faster than a glacier while there at it that'd be great.

I was kind of thinking about this.

Let's hypothetically assume we have some sort of AGI and we can ask it to write programs and text and nothing else.

Is there anyone on this planet who would think that they don't need to look at the generated code? I mean imagine a manager simply feeding in tickets and getting a finished application out without ever knowing how it was produced.

The application is business critical and any kind of mistake could ruin his business which puts the manager at complete mercy of the AI.

Now you might say that this happens with humans as well but when humans cause problems we let other humans review and test their code.

AI causes problems? Let's add more humans. Wait a minute...

I'm leaning towards using it for things I already know exactly how to do -- including a very clear idea of the result. In these contexts, it can save some mental workload / time.

> everything it generates can be pretty quickly verified and validated by another machine

It can be verified in a sense that it builds, but that doesn't mean that it actually does what you asked it to do, or that it does it on all valid inputs. The worst bugs to track down are silent logic bugs.