Re:Show HN: Whole Git repo was made with ChatGPT
by bcrosby95 on Sunday December 04, 2022 @06:09PM
(#33859054)
I was experimenting with it a bit while building an RPG combat system.
One thing I noticed is its very subservient. I pointed out a bug and it fixed it. But then I immediately did the reverse - I asked it to re-introduce the bug (by stating a fact about the existing program which was incorrect), and it was happy to re-introduce the bug I had it squash a minute ago.
This particular example was related to the hit chance of an attack. It was rolling 1-100. But then when modifying that chance by an NPC's statistics, it first multiplied that statistic by 0.01. It couldn't connect the fact that the rest of the program was already using whole number percentages (1-100) with a statistic modifying your chance to hit, and had decided to use 0-1.0 for that.
So be careful about how "smart" you think it is. It can churn out code. But it will also introduce bugs if you ask it to, and at least in this experiment would not assert itself in situations where my stated understanding of the program as written was wrong.
One thing I noticed is its very subservient. I pointed out a bug and it fixed it. But then I immediately did the reverse - I asked it to re-introduce the bug (by stating a fact about the existing program which was incorrect), and it was happy to re-introduce the bug I had it squash a minute ago.
This particular example was related to the hit chance of an attack. It was rolling 1-100. But then when modifying that chance by an NPC's statistics, it first multiplied that statistic by 0.01. It couldn't connect the fact that the rest of the program was already using whole number percentages (1-100) with a statistic modifying your chance to hit, and had decided to use 0-1.0 for that.
So be careful about how "smart" you think it is. It can churn out code. But it will also introduce bugs if you ask it to, and at least in this experiment would not assert itself in situations where my stated understanding of the program as written was wrong.