The article compares verified responses on r/AskDocs (yeah, a subreddit) and those from ChatGPT. That's it. How is its coding compatibility even remotely relevant? It's like saying "Excel is bad in editing photos, so it must be a bad spreadsheet software as well."
This is disingenuous. OP is right. ChatGPT is mostly inaccurate and contextless by nature.
It always produces something confidently so it's very easy to think it's the right answer.
Now, it can be extremely useful for any task that can be easily verifiable and you don't know the syntax, how to approach something, etc. Because any decent software developer can use it as part of the prototyping or brainstorming and get wherever they want to go in coordination with ChatGPT.
What you can't do is assume it's "the expert". You are the expert and you're the intelligent one and the chat generates potential useful things.
That's on verifiable stuff. On other things it can be so laughably bad it's impressive how much "everyone" (as in articles, hype, HN users who downvote criticism as being from luddites) pushes it as something that is not: an AI that can think and be relied upon and any inacuracy will just "get better with time" aa opposed to "the model and also the politics around it" doesn't have a path to get to that idealisation that is being sold.
It's a tool, but it's usually sold as better than it is, like in this case, presumably with the intent of relying upon it to save cost in some key integration point. The problem is that it mostly won't work and comparing it to bad humans or worse integrations doesn't show the fundamental low ceiling that it has.
I think people with authoritarian mindsets (and I don't mind Left or right but inherent trust in authority) easily want this to be a source of truth they can magically use, but there's no path for that to be true. Just to appear true.
Yes. The failure mode of a human and chatgpt are nothing alike — I am far more experienced in spotting beginner mistakes in code reviews, then seemingly good, but actually illogical bullshit code generated by LLMs.
I have never had it produce non-trivial, novel code that was correct, so I mostly use it as a search engine instead.
Either that or the problems they're solving are nothing boilerplate couldn't handle.
> if it doesn't know well a person it simply makes up things
Asking it for factual information about a subject can be a bit hit/miss depending on the subject. Better to use bing chat, because it will use info from the web to inform the response
> if I ask code examples or scripts, most of the time they are wrong, I need to fix them, they contain obsolete syntax etc...
How wrong? More wrong than having a junior or mid level developer contributing code?
Think about it a different way: you just gained an assistant developer that writes mostly correct code in seconds. Big time saver.
Also: if you want it to use a particular code style etc, give it few shot examples.
> if I'm asking a question, I'm expecting being asked for more context if the subject is not clear
Then you need to tell it that in you prompt: "if the subject isn't clear, ask me some clarifying questions. Don't respond with your answer until I have answered your clarifying questions first". Or: "ask me 3 clarifying questions before answering" to force it to "consider" how well it "knows" the subject first.
ChatGPT isn't an AI in the sci fi sense of the word. It's a language model that needs to be prompted the right way to get the results you want. You will get a feel for that the more you use it.