For me they save a lot of time on research or general guidance. But when it comes to actual code - not really useful.
I don’t get good output, not when trying to get code that matches a detailed spec (which includes the languages I wish to use, the structure of the APIs, and the libraries I think might be useful) so your suggestion that we’re “just wrong” and your claim that the tools can be used for coding “complex tasks” is difficult for me to swallow.
I’ll admit that perhaps I’m not using ‘em right but that’s why I’m here—to get advice on /how/ to use ‘em correctly; to date, the implications found in the advice I have received is to: - limit my scope to strict HTML/JS (not web/ui frameworks), or - limit the size of the project to a handful (less than ten) of very short files, or - limit my scope to code translation only, or - limit the size of my chat sessions.
Unfortunately, those limitations don’t fit the needs of my environment.
Yes, mistakes may happen. However, I’ve used it to translate a fairly complex MIP definition export into a complete CP-SAT implementation.
I use these models all the time for complex tasks.
One major thing that is perhaps not immediately obvious is that the models are only good at translation. If I give it a really good explanation of what I want in code or even English, and ask it to do it another way or implement it with specific tools, I get pretty good output.
Using these to actually solve problems is not possible. Give it a complex problem description with no instructions on how to solve it, and they fail immediately.