You'd have a much better time just Googling those asks and re-using a working examples from SO or GitHub. Which is ironic, given how ChatGPT is supposedly trained on those exact things.
I'm wondering how come we're both getting such vastly different results. Maybe your bar is just lower than mine? I don't know. I'm honestly shocked at the contrast between the PR given to ChatGPT, and the results on the ground.
Try this simple ask (the results of which you'll find plastered everywhere): produce a Python function that decodes a Base64 string and prints the results, without using any "imports" or libraries. Every single output I got back was embarrassing garbage, and I gave it something like 15 shots.
All that in mind, I'd be lying to say I'm not more than a little concerned with the progress from 3.5 -> 4. I'm only two years into my career and my fingers are crossed that it won't significantly impact the market for devs for as long as possible.
Here are some sample prompts that I tried and got full working code for:
- "write pytorch code to train a transformer model on common crawl data and an inference service using fastapi"
- "write react native code for a camera screen that can read barcodes and look them up using an API and then display info for matched results in a widget under the camera view"
- "write react code for a wedding website"
- "write code to deploy a django website on GCP using terraform and kubernetes"
- "how do I dockerize the app, it uses pytorch and faiss, also push it to a container registry"
- "implement a GPT style transformer model in pytorch", "write a training loop for it with distributed support and fp16"
- "how would you implement reinforcement learning with human feedback (RLHF)", "can you implement it in pytorch"
- "write code to compress a model trained in pytorch and export for inference on iOS"
- "how would you distill a large vision model to a small one"
- "what are the best CV architectures for mobile inference?"
For all of these it gave me code that was 95% usable, all in under 15 minutes, and which would have taken me a week or two to do on my own.