Re:Show HN: Whole Git repo was made with ChatGPT
by wizeman on Sunday December 04, 2022 @06:25PM
(#33859198)
> There is no incentive for AI to be correct, nor any incentive for it to become more correct.
Are you saying that for current AI, or also for future AI?
I'm not an expert, but I think reinforcement learning is a pretty well-established AI technique at this point, which does give an AI incentive to be correct and become more correct.
I've heard rumors OpenAI is even doing that with ChatGPT, based on human feedback (with the thumbs up/down button).
> Strong No Hire. And if you hire them and stick me on code-review duty, I quit.
I mean, I agree. I wouldn't hire ChatGPT to replace a programmer currently, just like I wouldn't hire many humans, even many who claim they are programmers/engineers.
But at this rate, I might just hire a future version of it :)
To wit: I asked ChatGPT to write me a function to generate Hanukkah dates. It spewed a couple of facts about Hanukkah, and the generated function just returned the 8 days following Christmas. I said it was wrong, and it said "you're right!" and then gave me another wrong function (always returning March 25th, since it's actually in the 3rd month of the Jewish calendar, duh). I told it that it was wrong again and asked specifically for a Jewish to Gregorian calendar conversion function, and it gave me some more code that I have no idea is correct or not. So I asked for a table of 10 test cases of Jewish dates and their corresponding Gregorian dates, and not a single one was valid.
I actually have worked with some remote contractors who pull this kind of crap, just throwing garbage over the wall repeatedly until I give up because I just don't want to deal with them anymore. And no, they never get any better. They are not junior devs you can mentor or train or whatever. Their whole model of work is to generate something complicated enough to seem plausible and then bullshit you into accepting it so they can get paid. I have to inspect every line of code they write and do all the diligence myself anyway, and this is more time-consuming and less fun than just doing it myself in the first place. At best it's a nerd-snipe motivation in a "wrong answer on the internet" kind of way.
Strong No Hire. And if you hire them and stick me on code-review duty, I quit.