I was skeptical but impressed, too. I created a .py file that started with a comment something like:

  # this application uses PyGame to simulate fish swimming around a tank using a boid-like flocking algorithm.

and Copilot basically wrote the entire application. I made a few adjustments here and there, but Copilot created a Game class, a Tank class, and a Fish class and then finished up by creating and running an instance of the game.

Worked pretty well on the first try. It was definitely more than I expected. I wish I had committed the original to GitHub, but I didn't and then kept tinkering with it until I broke it.

> I wish I had committed the original to GitHub, but I didn't and then kept tinkering with it until I broke it.

Isn't it deterministic, so you can just re-enter your original input and get your original version back?

Good question! I'm not sure I can get back to exactly the same input I originally used. Part of the experimenting I did involved erasing some of the original comment and altering it to see how it would impact the generated code.

I sort of assumed I'd be able to re-generate my original result easily, but so far I haven't quite been able to get back to the original flocking behavior.

Why was this downvoted? I had the same doubt.

Unsure, because I thought it was a fair question. I ended up replying. The short answer is that I'm not 100% sure what my original input was.

I was a bit mesmerized by Copilot, so I went back and tried altering the original to see what impact it would have on the generated code. Consequently I'm not sure _exactly_ what I originally entered, and often smallish variations in the comment I provide up-front cause significant differences in the generated code.

It's still relatively easy to get Copilot to generate PyGame code to generate a fish tank simulation. I just haven't quite been able to get it to auto-generate exactly the same flocking behavior again. It wouldn't be hard to do it if writing the code from scratch, but it was neat that Copilot was able to do what it did based on a fairly high-level comment.

I wonder if we can make it useful for refactoring. Like let's say I change the second argument of a function from int to long, then I do the same for another function. Can autopilot "understand" what I'm doing, and apply the same change to the rest of my source file?

Or more general. E.g. I'm in an graphics editor and move one shape so it touches a line, then another shape in the same way, can an AI understand what I'm trying to do and make all the shapes touch the line?

Had the same thought!

The OpenAI API has davinci-codex, which could probably be trained for refactoring, but it's private beta...

I think this is potentially a great use case, especially "code cleanup" tasks. Train it on several examples of messy code and clean code. I think it would have good results.

At least for Java, it can't do that. It only adds new code below the one you are typing. You can add a comment and it tries to "code" that comment.

I've tried it and half the time it's like "god damn this is the future" and the other half it's just repeating garbage in a syntactically correct way. It does particularly poorly for unstructured tasks like writing new functionality or classes.