Same here and GPT-4 was definitely a noticeable improvement.

You know, I thought so but then recently I asked it to code me a function to do a financial calc and it just didn't get there at all. It gave me code but it was really poor and didn't do close to what I wanted.

But when I gave it my code it did generate useful data for unit tests. So that's pretty cool.

I’ve found that whenever it gets something really wrong, it’s either corrected by a follow up prompt (telling it that its wrong helps), or rephrasing the question.

Nope, not my experience - unless it's a trivial problem. In my experience it almost always gets stuck in a loop where it's rotating between two or more incorrect implementations. It isn't doing iteration - it's just looping. Even if I point out the problem directly, it will just say "sorry, you're right - here's the correction!" and the correction has the same response from an earlier item that I pointed out had a problem!

Used to get that a lot with GPT-3.5. But GPT-4 has been more reliable if you rephrase your prompt.

Have not had that experience with GPT-4. But that is almost unusable for me anyway because it's crazy slow (during US working hours anyway, it's faster at night) and the rate caps get hit before we get anywhere.