The Inherent Limitations of GPT-3

Re:The Inherent Limitations of GPT-3 by rpeden on Monday November 29, 2021 @05:21PM (#29386037)

I was skeptical but impressed, too. I created a .py file that started with a comment something like:

  # this application uses PyGame to simulate fish swimming around a tank using a boid-like flocking algorithm.

and Copilot basically wrote the entire application. I made a few adjustments here and there, but Copilot created a Game class, a Tank class, and a Fish class and then finished up by creating and running an instance of the game.

Worked pretty well on the first try. It was definitely more than I expected. I wish I had committed the original to GitHub, but I didn't and then kept tinkering with it until I broke it.

[ Parent | on HN ]

Re:The Inherent Limitations of GPT-3 by amelius on Monday November 29, 2021 @07:07PM (#29387090)

> I wish I had committed the original to GitHub, but I didn't and then kept tinkering with it until I broke it.

Isn't it deterministic, so you can just re-enter your original input and get your original version back?

[ Parent | on HN ]

Re:The Inherent Limitations of GPT-3 by rpeden on Monday November 29, 2021 @09:12PM (#29388000)

Good question! I'm not sure I can get back to exactly the same input I originally used. Part of the experimenting I did involved erasing some of the original comment and altering it to see how it would impact the generated code.

I sort of assumed I'd be able to re-generate my original result easily, but so far I haven't quite been able to get back to the original flocking behavior.

[ Parent | on HN ]

Re:The Inherent Limitations of GPT-3 by muxator on Monday November 29, 2021 @08:56PM (#29387887)

Why was this downvoted? I had the same doubt.

[ Parent | on HN ]

Re:The Inherent Limitations of GPT-3 by rpeden on Monday November 29, 2021 @09:17PM (#29388055)

Unsure, because I thought it was a fair question. I ended up replying. The short answer is that I'm not 100% sure what my original input was.

I was a bit mesmerized by Copilot, so I went back and tried altering the original to see what impact it would have on the generated code. Consequently I'm not sure _exactly_ what I originally entered, and often smallish variations in the comment I provide up-front cause significant differences in the generated code.

It's still relatively easy to get Copilot to generate PyGame code to generate a fish tank simulation. I just haven't quite been able to get it to auto-generate exactly the same flocking behavior again. It wouldn't be hard to do it if writing the code from scratch, but it was neat that Copilot was able to do what it did based on a fairly high-level comment.

[ Parent | on HN ]