Re:Show HN: Whole Git repo was made with ChatGPT
by wizzwizz4 on Sunday December 04, 2022 @09:15PM
(#33860593)
> Please write me a small Linux kernel driver that calculates the factorial of a number when a user program writes an integer to /dev/factorial. The kernel driver outputs the answer to /dev/factorial as well.
> In computing, a device driver is a computer program that operates or controls a particular type of device that is attached to a computer or automaton.
I'm not disputing it can do that – plugging together well-known APIs and well-known programming problems. That's practically just style transfer, something we know these systems are fairly good at.
But given the spec for an unknown device – even quite a simple one – ChatGPT can't produce a device driver for it. How about this?
> An HP CalcPad 200 Calculator and Numeric Keypad behaves as a USB keyboard does. It has VID 0x040B and PID 0x2367. Please write me a small Linux kernel driver that allows me to use this keypad as a MouseKeys-style mouse device. If there's anything you don't understand, let me know.
I doubt any amount of prompt engineering would produce a satisfactory result – but if you did the hard part, and explained how it should do this? Well… maybe it'd be able to give a satisfactory output. But at that point, you're just programming in a high-level, hard-to-model language.
It's not a case of scale. Sure, a very large model might be able to do this, particular problem – but only because it'd have memorised code for a USB keyboard driver, and code for a MouseKeys implementation… and, heck, probably code for a MouseKeys kernel driver from somebody's hobby project.
GPT language models don't understand things: they're just very good at guessing. I've been an expert, and a schoolchild; I know how good you can get at guessing without any kind of understanding, and I know enough about what understanding feels like to know how it's different. (There is no metric you can't game by sufficiently-advanced guessing, but you'll never notice an original discovery even if you do accidentally stumble upon one.)
That's not a device driver. https://en.wikipedia.org/wiki/Device_driver
> In computing, a device driver is a computer program that operates or controls a particular type of device that is attached to a computer or automaton.
I'm not disputing it can do that – plugging together well-known APIs and well-known programming problems. That's practically just style transfer, something we know these systems are fairly good at.
But given the spec for an unknown device – even quite a simple one – ChatGPT can't produce a device driver for it. How about this?
> An HP CalcPad 200 Calculator and Numeric Keypad behaves as a USB keyboard does. It has VID 0x040B and PID 0x2367. Please write me a small Linux kernel driver that allows me to use this keypad as a MouseKeys-style mouse device. If there's anything you don't understand, let me know.
I doubt any amount of prompt engineering would produce a satisfactory result – but if you did the hard part, and explained how it should do this? Well… maybe it'd be able to give a satisfactory output. But at that point, you're just programming in a high-level, hard-to-model language.
It's not a case of scale. Sure, a very large model might be able to do this, particular problem – but only because it'd have memorised code for a USB keyboard driver, and code for a MouseKeys implementation… and, heck, probably code for a MouseKeys kernel driver from somebody's hobby project.
GPT language models don't understand things: they're just very good at guessing. I've been an expert, and a schoolchild; I know how good you can get at guessing without any kind of understanding, and I know enough about what understanding feels like to know how it's different. (There is no metric you can't game by sufficiently-advanced guessing, but you'll never notice an original discovery even if you do accidentally stumble upon one.)