It wasn't really anything I couldn't have written in a half hour or so but it was so much faster. The real kicker is that by default chatgpt wrote Rspec and I was able to say "rewrite that in minitest" and it worked.
I've found Copilot useful when writing greenfield code, but very unhelpful generating code that uses APIs not popular enough to have significant coverage on StackOverflow. Even if I have examples of correct usage in the same file it still guesses plausible but wrong types.
I haven't bought GPT 4 but I'm curious if it's much better at this.
If you ask for something impossible in a library it will also frequently make up functions or application settings. If you ask for something obscure but hard to do, it might reply that it's impossible but it is possible if you know how and teach it.
I sort of compare prompt engineering to Googling - you sometimes have to search for exactly the right terms that you want to appear in the result in order to get the answer you're looking for. It's just that the flexibility of ChatGPT in writing a direct response sometimes means it will completely make up an answer.
There's also a limitation that the web interface doesn't actually let you upload files and has a length limit for inputs. For Copilot, I'm looking forward to Copilot X: https://www.youtube.com/watch?v=3surPGP7_4o
Prompt:
fn encode(value: Foo) {
capnproto::serialize_packed:: serialize_message(value);
}
fn decode(input: &[u8]) {
capnproto::serialize_packed:: deserialize_message(input);
capnproto::PackedMessageDeserializer::deserialize(input)> In summary, you should use read_message when working with packed messages and deserialize_message when working with unpacked messages. Make sure you choose the appropriate serialization and deserialization functions based on the format in which your messages are stored or transmitted.
Google was surprisingly little help on the topic. At best it pointed me to https://capnproto.org/encoding.html#packing which covers the general idea but glosses over parts.
The problem with ChatGPT is you can never be certain or confident that its answers are correct. It’s as useful as rolling dice in guessing a number sometimes. With the training from the internet, the dice are loaded, but the answer is still likely to be wrong because it just assembles words together.
I see the future as writing test cases (perhaps also with ChatGPT), and separately using ChatGPT to write the implementation. Perhaps we will just give it a bunch of test cases and it will return code (or submit a PR) that passes those tests.
It’s certainly not quicker answers, right?
Even personal projects, where I'm learning new languages and libraries, I've found that the code that gets generated in most cases is incorrect at best, and won't compile at worst. So I have to go through and double-check all of its "work" anyway- just like I'd have to do if I had a junior engineer sidekick who didn't know how to run the compiler.
I think for the work problems, if our company could train and self-host an LLM system on all of our internal code, it would be interesting to see if that could be used to assist building out new features and fixes.
Not cheap, though. I think most companies will end up hosting an LLM on local code rather than using OpenAI on Azure, but for now, there is no equivalently-capable replacement.
Many people are still sleeping on how useful LLMs are. There's a lot of related things to be skeptical about (big promises, general AI, does it replace jobs, all the new startups that are basically dressed up API calls...) but if you do any kind of knowledge work, there's a good chance that you could it much better if you also used an LLM.