More of the same.
The danger is that the content generated by ChatGPT is very coherent, but technical details are frequently wrong. It's going to become increasingly difficult to separate the signal from the noise on the internet.
I think AI is going to force us on some level to reclaim some of our communication to the physical world, if for no other reason than it's going to be one of the only ways we can truly verify that we're talking to another person.
Yikes. Now I imagine a totally unqualified candidate for literally anything feeding interview questions to the chat bot and I don't like it at all.
My single worst interview experience was an on-site five hour marathon of whiteboard coding, with a grumpy senior insisting that the code on the whiteboard be syntactically correct. Nothing screams "we want unthinking ticket crunching machines" like optimizing for candidates willing to focus on writing code by hand for hours on end.
Naturally, I rejected the follow-up interview, but I fear that more companies now are going to demand this nonsense.
Side note: in my personal example, the whiteboard session wasn't the reason I turned them down; I asked every person on the team roughly how many hours a week they worked and not one of them answered the question (instead redirected the conversation towards "you can work from home sometimes!" type answers).
Since then, however, I have rejected other companies trying to ape this style flat out. A short half hour, fine. Five hours? Pound sand, I say.
Maybe eventually there will be a public understanding that only a foolish child would read a reddit frontpage and expect human content. It will all be auto generated to trap people, like a gatcha game.
I just don't see how AI won't be used to manipulate these kinds of barriers. Once AI reaches a point where it can truly equal or surpass humans in terms of operating online, how are we going to prevent it from circumventing those barriers? If AI can generate code from a chat prompt, who's to say it can't solve a captcha in the future? And once that happens (because we all know it will at some point), how are we going to be able to differentiate?
Do you see the problem now?
For example, editors and reviewers for academic journals / conferences will likely see a deluge of AI-generated "scientific" papers. Their time is limited, and odds are that more papers with incorrect information will slip through the peer review process.
To be clear, peer review today certainly isn't perfect at catching bad papers. But AI generation has the potential to exaggerate the problem.
What we consider before using anonymous sources: How do they know the information?
What’s their motivation for telling us?
Have they proved reliable in the past?
Can we corroborate the information they provide?
Because using anonymous sources puts great strain on our most valuable asset: our readers’ trust, the reporter and at least one editor is required to know the identity of the source. A senior newsroom editor must also approve the use of the information the source provides.
https://dailysceptic.org/2022/06/08/fake-science-the-threat-...
The sad thing is it doesn't take a ChatGPT level intelligence to beat scientific peer review. Journals routinely publish papers that are completely auto-generated gibberish. A simple generative grammar or template splicer is apparently enough. These are articles that are immediately visible as the work of a program at first glance, they wouldn't make it past even the most amateur blog or student newspapers, yet they surface in their thousands in journals that are supposed to be the epitome of accurate knowledge!
Worse, the journal publishers are doing nothing about it. Their current approach to trying to fix the problem is to try and use the work of random CS academics to make "spam filters" for paper submissions. The more obvious solution of having editors and reviewers who actually read scientific papers before they are published appears to be rejected out of hand.
The better that computers get at generating content and behaving in ways that only humans used to be able to is going to make it harder to determine if a source is human or not.
More and more content that gets created by machine that is of low quality will get in the way of its own future training.
There will eventually be less and less human made content to train from.
I've been thinking along these lines a lot recently - it seems as though in every field which AI enters, it causes us to return to some extent to the physical world.
But yes, will be "interesting" to observe, how such a new wave will make theinternet worse.
And not only that, what's most amazing is that it can exceed humans at abilities that many years ago, most people thought only humans could accomplish (creativity, creating art, writing poems/literature, etc). And these models can only improve in the future!
If I have a robot that talks like a human, including answering questions like a human, behaves like a human in any way that is important... wouldn't this "tool" be intelligent, just like a human?
How are robots different than ChatGPT except having a physical presence and being able to manipulate the world physically?
Or are humans so special to you (or intelligence so subjective) that you can't possibly answer "yes" to the above statement about a robot being intelligent?
The analogy is more like: if robots can write like a human, then robots have the ability to write, and saying otherwise is just applying a double standard.
Similarly, if a robot can behave as intelligently as a human, then such a robot is intelligent and I don't see how anyone can argue otherwise without logical fallacies.
but are we saying robots can behave intelligently as a human?
I believe that ChatGPT can behave more intelligently than most humans at many important and useful tasks already.
I also believe that it behaves less intelligently than most humans at many tasks that are also important and useful.
For some other tasks, it behaves at more or less the same level as a human.
Moreover, I believe that the gaps between what models like ChatGPT can do and what a human can do are closing extremely rapidly.
To wit: I asked ChatGPT to write me a function to generate Hanukkah dates. It spewed a couple of facts about Hanukkah, and the generated function just returned the 8 days following Christmas. I said it was wrong, and it said "you're right!" and then gave me another wrong function (always returning March 25th, since it's actually in the 3rd month of the Jewish calendar, duh). I told it that it was wrong again and asked specifically for a Jewish to Gregorian calendar conversion function, and it gave me some more code that I have no idea is correct or not. So I asked for a table of 10 test cases of Jewish dates and their corresponding Gregorian dates, and not a single one was valid.
I actually have worked with some remote contractors who pull this kind of crap, just throwing garbage over the wall repeatedly until I give up because I just don't want to deal with them anymore. And no, they never get any better. They are not junior devs you can mentor or train or whatever. Their whole model of work is to generate something complicated enough to seem plausible and then bullshit you into accepting it so they can get paid. I have to inspect every line of code they write and do all the diligence myself anyway, and this is more time-consuming and less fun than just doing it myself in the first place. At best it's a nerd-snipe motivation in a "wrong answer on the internet" kind of way.
Strong No Hire. And if you hire them and stick me on code-review duty, I quit.
Are you saying that for current AI, or also for future AI?
I'm not an expert, but I think reinforcement learning is a pretty well-established AI technique at this point, which does give an AI incentive to be correct and become more correct.
I've heard rumors OpenAI is even doing that with ChatGPT, based on human feedback (with the thumbs up/down button).
> Strong No Hire. And if you hire them and stick me on code-review duty, I quit.
I mean, I agree. I wouldn't hire ChatGPT to replace a programmer currently, just like I wouldn't hire many humans, even many who claim they are programmers/engineers.
But at this rate, I might just hire a future version of it :)
1) is the end result useful/valuable?
2) did the GPT-based 'coding' process provide any of the following benefits: [reduce cost | reduce time to market | increase quality]?
Do you mean?
Is it literally as good as a human software engineer? No, but it's also better too. I doubt ChatGPT could debug as effectively as a veteran software engineer (... In fairness, most humans can't either.) It can debug pretty decently, but there's still work there. That said, the breadth of knowledge encoded in a language model is stunning. I'm pretty sure you can't just regurgitate an implement of the discrete cosine transform in Rust without at least pulling up Wikipedia, but ChatGPT can, because well. It doesn't have to pull it up.
I still don't think ChatGPT is ready to replace human programmers. It may be a long time before we have general enough intelligence to replace knowledge work meaningfully with AI. However, if you think it's not happening ever, because machines are not "intelligent" based on some set of goal posts, I've got bad news: that's not part of the job listing.
It's easy to laugh at MSN publishing articles written by GPT; that's just stupid. However, at some level you have to admit that the input to ChatGPT is almost as high level as directives from project managers, and the output is almost low level enough to simply input directly into source control. That leaves very little to the imagination for how this could quickly spiral out of control.
"Write a function that sorts the elements of a c string backwards, in C." "Add a flag to the sort function which makes it sort the string forward." "Could you write a endian swap function for double argument in PIC assembler?" "Could you write a binary sort in PIC 8-bit assembler" "Write a Javascript function that prints the day of the week together with some Chinese words of wisdom."
It had no problem doing any one those. I ran them all, except the assembler ones.
The question is how good it is to process larger chunks of code and makes changes to it.
People thinking about becoming programmers might need to rethink their plans if this one improves ...
EDIT: Oh dear. I introduced bugs in its sort code and it found them and explained what they did.
What do you think coders do?
Did they learn to code by themselves, without ever looking at any preexisting code, or what?
I've seen enough programmers who can't program, but most of them can at least be taught to program. (The look on their faces when they finally start to get it… It's one of the best things.) Most people working on the Linux kernel can actually program.
Most software engineering work is just plumbing existing libraries together, according to a specification given by a non-programmer. The hard part is translating the business requirements into something that a computer could understand; the exact syntax can be handled by a search engine, or a predictive text algorithm.
ChatGPT can't write a kernel device driver, and it can't act as a no-code tool for non-programmers. Those are the hard parts.
> “To tapping machine with hammer, 10s.; to knowing where to tap it, £10; total, £10. 10s.”
Oh, do I have news for you then.
Look at what I just did with ChatGPT in 30 seconds (and I did not cherry-pick, these were the first answers I got!):
https://gist.github.com/wizeman/b269be035308994be745025fc337...
Now to be fair, the code is probably not totally correct, as probably there are parts still missing/wrong and there might even be compilation errors or other problems.
But here's the important part: you can tell which errors or problems you've observed and ChatGPT will fix these problems for you. Exactly like what a programmer does.
And sure, it cannot yet do this at scale, such as in implementing a huge kernel driver like a GPU driver.
But at this rate, give it a few years and an improved version might just be able to do anything a programmer does, perhaps even autonomously if we allow it to interact with a computer like a human does.
Weird flex, as that code is like 90% boilerplate[1]. Everyone was freaking out about Copilot and no one seriously ended up using it because it just generates buggy (or copyrighted) code. It can't even handle writing unit tests with decent coverage (which is arguably the most repetitive/boring software engineering task).
[1] https://github.com/ngtkt0909/linux-kernel-module-template/bl...
Isn't 90% of code boilerplate anyway?
Also, didn't ChatGPT generate more than just the boilerplate?
Didn't it interpret what I wanted and generated the code for computing the factorial as well, as well as modifying the boilerplate (e.g. the kernel module name, printed messages, function names, the module description, ...) so that it matches what the kernel module is supposed to do? Which is exactly what a human would do?
Aren't you also missing the fact that I gave it a 2-sentence instruction and it "understood" exactly what to do, and then did it? Like a human programmer would do?
Which, in sum, is totally the opposite of what you were claiming?
> Everyone was freaking out about Copilot and no one seriously ended up using it because it just generates buggy (or copyrighted) code.
Don't most programmers also generate buggy code at first? Don't they iterate until the code works, like what ChatGPT does if you give it feedback about the bugs and problems you've encountered?
Also, Copilot and ChatGPT have different levels of capabilities, don't assume just because Copilot can't do something, that ChatGPT can't. ChatGPT is clearly a big step forward as you can clearly see from how everyone is freaking out about it.
Finally, don't assume that these models are never going to improve, ever again.
[citation needed]
I mean, I’ve seen people claiming to use it and that it has significantly accelerating their work. On what are you basing the conclusion that it has no serious use?
Here's example of tiny webapp I wrote recently: https://github.com/vbezhenar/pwgen/blob/main/pwgen.html
Of course it wasn't Copilot writing it, but it definitely helps with boring parts. Like I'd write
const charactersElement = document.getElementById('characters');
It's like having stupid but diligent assistant who's happy to copy&paste&adapt parts of code.
I can't claim that I often use fully generated Copilot functions. Sometimes I do, often with significant rework, but that's because, as I said, I'm very picky.
I paid GitHub $100 already and don't regret it.
Though I think that Copilot has plenty of features ahead.
For example finding obvious issues in the code would be very useful. Like typos.
Another issue with Copilot is that it only generates new code. Imagine that I need to edit 10 similar lines. I edit one line and I'd like Copilot to offer other edits.
Also UI is lacking. Like it generates 10 lines but I only like first line. Now I have to add 10 lines and delete 9.
But I'm sure that those are obvious directions.
What's your point, that ChatGPT wouldn't know how to do that, especially if the kernel maintainers gave it such feedback?
I thought it was clear that it can in fact do that (sometimes by asking clarifying questions, like a human would).
I think some of the major things missing in ChatGPT is the ability to interact with a computer directly (including the compiler and checkpatch.pl, and using files for information storage instead of a limited N-token context), as well as interacting with humans by itself (e.g. via email).
And sure, it would still have very limited capabilities in many ways, don't get me wrong, as I don't think it could replace a programmer at this point.
But I think the gaps are closing rapidly.
(about the fact that a bot handles the "creative" work of writing the device driver while a human will do the repetitive work of reindenting)
(sorry, I should have used /s)
Please do tell, how is it hilariously wrong?
It seems to have written a factorial function just like it should, it implemented the logic to read the integer from /dev/factorial when a user-space program writes to it, and then it writes the result back to /dev/factorial, and it also returns the number of bytes written correctly.
Which was the entire point of the exercise. Also note that ChatGPT itself said it was just a sample and it might be incomplete.
I noticed it has a bug, because it reads `len` bytes instead of `sizeof(int)` bytes, but a programmer could have made the same mistake. Perhaps some additional logic to handle reading multiple integers and writing multiple answers could be written, but that would be a further iteration of the code, not the initial one that I would write.
If that is hilariously wrong, then I would also be hilariously wrong. And I'm not just some random web developer, I actually wrote Linux kernel code professionally for years (although, that was a very long time ago).
So, maybe it got some details wrong, but I could conceivably also get those details wrong until I tried to compile/run the code and see what was wrong.
> I would be more impressed if you used chat GPT to guide you through reverse engineering a piece of hardware and implementbing a driver for it.
Yes, I would be more impressed with that as well. Perhaps someone will do that sometime. Even if not with ChatGPT, perhaps with a future version of it or a similar model.
Also as you noted, the logic related to handling `len` vs `sizeof(int)` is... curious.
While I find some ChatGPT examples impressive, this one isn't very enlightening. The block device structure and the factorial itself are fine, but those are likely in the training set as there are various tutorials online. But the combination of the factorial function into the block device is pretty awful (though I could imagine a novice doing it like this).
Both the OP and this driver kind of fit into this pattern, it looks like someone asked a novice programmer to do something using X and Y, so he read various few sample code and tutorials on X and Y and tries to put something together as best as it could, but the "plumbling" between X and Y (the part that wasn't in the tutorials) doesn't really work.
Yes, you're probably right. It's been many years since I've written Linux kernel code and also, I've never implemented such a driver, so I missed that.
I asked ChatGPT: what is wrong with this Linux kernel code (and fed it the dev_write() function), and this is what it answered (on the first try):
> There are a few issues with this Linux kernel code:
> 1. The function dev_write() is declared as returning ssize_t, but the return value at the end of the function is sizeof(int). ssize_t and size_t are different types, so this could cause a type mismatch.
This is an interesting observation that you and I both missed.
> 2. The factorial() function is not defined in this code, so it is unclear what it does or where it is declared.
This is because I didn't feed it the factorial function, it was on a different chat session. One could guess what it does based on the name, but to be fair, it's not clear why such a function would be on the Linux kernel unless I gave it the full context.
> 3. The copy_from_user() and copy_to_user() functions are used to copy data between user space and kernel space. In this code, buffer is a pointer to user space, but input and result are local variables in kernel space. This means that copy_from_user() will copy data from user space into kernel space, but then copy_to_user() will copy data back from kernel space into the same buffer in user space, overwriting the original data. This is likely not the intended behavior.
I think this is basically what you pointed out.
> 4. The len parameter is not used in this function, so it is unclear why it is included as an argument.
lol. Actually, it is being used, incorrectly.. and it wouldn't be used if we did the correction that I mentioned. So yes, this is somewhat hilariously wrong.
And it's clear why it has to be included as an argument, but to be fair, ChatGPT has no context whatsoever except knowing it's Linux kernel code (because I used a different chat session), so it's conceivable that it couldn't figure out why the function prototype has to be that way.
> Overall, this code is not well-written and would benefit from further clarification and revision.
It was you who wrote it, dumb-ass! :)
(Interestingly, ChatGPT can be compared to someone with Alzheimer in the sense that it can't remember conversations that happened in different chat sessions).
> This is an interesting observation that you and I both missed.
Hah. Call me when you find an architecture where ints use over half the addressable memory.
I mean, I get your point if it's a joke :) But I think the AI was just pointing out that you'd get a compiler warning because of the type mismatch in signedness (is this even a word?).
It copies from user, and then copies to user, back into the const it got. I don't even know if this would compile but certainly nobody would expect the buffer they just wrote to a file to suddenly end up re-written with a "reply" when using a special file.
I found that logic very funny.
Edit: To elaborate, the actual code to do this properly would have to allocate some memory when the file was opened, this memory would hold the reply once a question was written. Then when the answer was read it would have to respond with that in there. Finally when the file is closed it would have to deallocate.
This code is quite far from accurate, the issue I have with the concept that even if this bot was 99% accurate, C is not a "try it and see" language. If you apply "try it and see" techniques to C you just get subtle undefined behaviour which you might not notice until someone uses it to remotely execute code on your machine. Really I am not so sure that even humans can be trusted to write C but at the very least I am well aware of all the instances where C is a minefield and can pay extra attention to those, if you simply look at the plethora of information on C out there to learn C and apply it to kernel development you won't automatically build these intuitions no matter how much code you read because an enormous amount of C out there is hideously wrong in many ways.
Final edit: Once you start looking at the details, it has managed to implement a bunch of buffer overflows to boot.
That's not a device driver. https://en.wikipedia.org/wiki/Device_driver
> In computing, a device driver is a computer program that operates or controls a particular type of device that is attached to a computer or automaton.
I'm not disputing it can do that – plugging together well-known APIs and well-known programming problems. That's practically just style transfer, something we know these systems are fairly good at.
But given the spec for an unknown device – even quite a simple one – ChatGPT can't produce a device driver for it. How about this?
> An HP CalcPad 200 Calculator and Numeric Keypad behaves as a USB keyboard does. It has VID 0x040B and PID 0x2367. Please write me a small Linux kernel driver that allows me to use this keypad as a MouseKeys-style mouse device. If there's anything you don't understand, let me know.
I doubt any amount of prompt engineering would produce a satisfactory result – but if you did the hard part, and explained how it should do this? Well… maybe it'd be able to give a satisfactory output. But at that point, you're just programming in a high-level, hard-to-model language.
It's not a case of scale. Sure, a very large model might be able to do this, particular problem – but only because it'd have memorised code for a USB keyboard driver, and code for a MouseKeys implementation… and, heck, probably code for a MouseKeys kernel driver from somebody's hobby project.
GPT language models don't understand things: they're just very good at guessing. I've been an expert, and a schoolchild; I know how good you can get at guessing without any kind of understanding, and I know enough about what understanding feels like to know how it's different. (There is no metric you can't game by sufficiently-advanced guessing, but you'll never notice an original discovery even if you do accidentally stumble upon one.)
Developing the Linux kernel is just plumbing already existing CPU instructions together...
Developing a CPU is just plumbing already existing electronic components together...
Developing an electrical component is just manufacturing with already existing materials...
No, that's actually the easy part.
The hard part is to translate the requirements into something that a computer can understand and a human can also easily understand at the same time. Because otherwise, it's a one-off program that can't really be changed afterwards anymore.
And that is the real challenge that I'm curious about: how good will ChatGPT be in not only helping me with small, trivial, issues, but with the big ones.
"ChatGPT, here's this small 500k line repository. I need to change the business requirements and make it so that when a user fails to login 3 times in a row, a captcha is being shown before they can attempt again."
And if ChatGPT then gives me a 10k line diff that would be terrible. It should really minimize the diff but still keep it understandable. THAT is what I would love to see.
ChatGPT can also learn about Commodore programming if you feed it the Commodore BASIC manual during training.
My guess is that it could learn even more/better if you'd allow it to interact with a Commodore 128 computer like you did when you were learning!
Perhaps the goal isn't to use AI to write up an entire repo, but to go through easy utils so the highly paid engineer can focus on the core logic, the core USP of the business.
In the hand of a newbie programmer with like one week of learning experience this thing will allow them to make usable programs. I don't think this thing will hit us with years of experience in the short term but entry level programmers might be in real danger of being flooded if OpenAI makes some "low code" IDE that just works and parses the whole set of code in the project when evaluating queries.
If you can't think of such a test, then it seems strange indeed to confidently claim that ChatGPT has failed to satisfy an untestable condition.
But if you can think of such a test, and any human fails it — does that mean the human who fails it isn't intelligent?
Searle's Chinese Room thought experiment argues with great precision why (solely) symbol-manipulating systems don't (and will never) think. The process of cognition is deeply internal and most likely involves self-reflection, internal states, and a sense of "I-ness" which seems to be unique to mental things (unlike Searle, I'm a substance dualist so I'm admittedly a bit biased). Ergo, I'd say, at the very least, an intelligent & thinking process would need to (also) operate on non-symbolic things (which rules out computational systems), maybe on qualia, but there's probably other primitives as well.
Thanks, I appreciate you pointing this out. Given that's the case, it might not actually be possible to reconcile our two views and we can just agree to disagree. (I was a physicist for ten years, so it's challenging for me to inhabit a position that contends that thought and matter are non-overlapping magisteria.)
I guess one question I might ask you is: how do you think substance dualism will affect your expectations about AI going forward? For example, is there any specific, measurable thing that you expect a future AI system absolutely will not be able to do? Or do any such "forbidden" things have to fall into categories like "thought", "feeling", etc. — i.e., the magisterium of the "mental", as opposed to the "physical" — under substance dualism?
For what it's worth, Searle is a naturalist (so he thinks brains are "just" meaty machines) and he would also disagree with you.
> For example, is there any specific, measurable thing that you expect a future AI system absolutely will not be able to do?
Not sure how you'd measure this, but I think sentience is most likely unachievable by AIs (at least how we're building them now). This is not without controversy, as while Roger Penrose would agree with me, David Chalmers would not. I'm not actually sure when Chalmers pivoted on his stance (haven't read him much since undergrad) because he was a bit more hardline in the early 2000s[1].
I definitely disagree that Searle's conclusion in the Chinese room experiment applies to the kinds of AI systems we're building today.
I agree that nobody to my knowledge has figured out how to measure sentience, so I don't think that fits under the "measurable" heading. And this does suggest to me — because this was your hand-picked example — that your position as a substance dualist doesn't affect your concrete expectations about what AI systems will actually be able to do in the future. Is that right?
Unless you're trying to say that today's AI systems do anything but manipulate symbols (which is imo patently obvious), I'm not sure how you can make that argument.
I take issue with this; imo its output looks exactly like what a neural network being fed terrabytes and terrabytes of natural language then recombining it would look like. But either way, you're making the same mistake: looking at behavior and affirming the consequent (namely: it outputs smart-looking text, therefore it must be intelligent). But this is a mistake. Its behavior implies nothing about underlying processes.
Oh btw you must’ve missed the post in which it was told it was a Linux shell and it mostly worked as one. Complete with recursively calling into a pretend API version of itself. I’m not calling that intelligence, but I’m not calling it regurgitation either.
That view is called "behaviorism" and is not really taken particularly seriously exactly because it's not very useful in truly understanding what is happening under the hood (which, as a curious species, we deem important). It's like not caring how electromagnetism works because the behavior of "the positive side of this chunk of rock is attracted to the negative side" is good enough.
Why is that a mistake? What other means do we have of assessing intelligence?
I think you need to move beyond Searle because he isn’t right.
I did a philosophy degree and smart people still very much take him seriously. If not him, take a look at Nagel or Chalmers. I think that his idea† that conscious experience is non-computable is probably right, especially given findings in the early 20th century that many things that we thought may have been computable ended up not being computable.v
† Not exactly his idea, though it nicely segueys into it.
I agree that some people take him seriously. Many people take religion seriously too. I don't take either him or religion seriously as true, but I do understand both his and religion's appeals.
That said, I do give a decent chance to aspects of the brain's problem solving ability having quantum properties and thus can not be calculated as efficiently on classical computers. We do not know this for sure, but evolution is incredibly inventive at finding efficient solutions. Still computable though, even on classical computers.
So, really, all that this tells you is that many people have this assumption.
Machine learning is modeled at least in small part in a way that it's thought the brain operates. I expect such things to be able to mimic human reasoning and intelligence, I just deny that there's any consciousness behind it -- there's no qualia, no phenomenology. In short, there's nothing that it's like to be ChatGPT. Even so, ChatGPT is able to mimic human intelligence, and I won't be surprised if (when?) it gets even better.
I also wonder: is there a chance you had more dialogue, but only presented this interaction?
> People thinking about becoming programmers might need to rethink their plans if this one improves ...
Very true, and this thought definitely crossed my mind as well. In the short term I imagine this is going to be like any other tool, it will increase your leverage, and make it possible for you to write code faster.
For example, I fed it 100 lines of a Django model and asked it to write some query code for me. Produced something that was possible faster than I could have. The few slight deficiencies it did have (optimizations) basically felt like me doing a code review.
It's nothing short of a phenomenal milestone.
[1] https://twitter.com/GuiAmbros/status/1599282083838296064
Can you see if it solves an advent of code problem whose solution hasn't been posted yet? If it did I will consider this an almost publishable breakthrough.
For example, start with "write a level generator in JavaScript which outputs pacman like levels encoded in ASCII which could be used as the basis for a pacman clone."
Then you'd say "Great, now write a pacman clone in JavaScript which takes the input of the previously generated level generator and takes interactive input on the console."
Then you'd have to ask it to generate the HTML and CSS. It won't do it all at once. But it will do it piece by piece.
The main thing is having a good idea of what the overall program structure will look like. If you have that, I think you can go pretty far with it.
Btw, in case anyone wants to see what working like this may look like, I recorded myself while doing that little experiment: https://youtu.be/TBMQIDMwI5E
Won't link it again so as to not get too spammy, but I posted a recording of an experiment I did playing around with ChatGPT in that way on a sibling comment if you want to check that out.
9 compile errors, some due to cargo.toml issues (it only generated main.rs). But mostly worked.
The post we’re discussing is also pretty impressive, a working lexer/parser
These models are amazing at regurgitation. They have no skill at reasoning and any solution outside their training set. They have a distinct disability for precision in anything remotely novel or large. And they will bullshit subtle bugs all over the place at scale.
Even a junior programmer would produce more value than the next generations of this, simply because they can take vague English instructions and actually make iterative progress across a huge codebase, test those changes for all sorts of vague constraints, and be precise. There’s no ability for that here.
It seems to be making the same kind of errors in code as we've seen in speech, you get a lot of plausible but subtly wrong hallucinated content that only someone who knows the subject can identify. It prefers probable output over matching the specifications so it'll add unnecessary branches just because it's seen them millions of times.
Among errors I've seen it make there is bad indexing, adding conditions that serve no purpose or assuming a variable has a specific structure when it doesn't.
I have a feeling though that tools like this will lower the skill threshold for programming by a lot.
I gave it this code I just made and asked it what it does and it figured it out:
public class mean_sink {
double sum;
long n;
public mean_sink(){
}
public void add(double d){
sum += d;
n++;
}
public double getMean(){
if(n == 0)
return Double.NaN;
return sum/n;
}
}
I dunno. This is insane anyway. It is really hard to grasp the consequences.
At the rate this is evolving that wont be a limitation for long.
My 14 yr old son thats taught himself to code in c# over the last few years is going to be put of a potential job before he cab be taught to code professionally.
This is quite frightening, the rate of evolution.
Do you write code close to the machine at $job?
I wanna try with more feature and UI heavy request.
I actually comment because of your edit in your response : you manually introduced bugs, reran the process, and it seen you broke it ( fine ) but was able to explain it to you in English ?
Where is that comming from ? Adjacent comments and git comments?
I fed it it's own sorting function but I added bugs to it and asked it to spot the bugs.
One thing I noticed is its very subservient. I pointed out a bug and it fixed it. But then I immediately did the reverse - I asked it to re-introduce the bug (by stating a fact about the existing program which was incorrect), and it was happy to re-introduce the bug I had it squash a minute ago.
This particular example was related to the hit chance of an attack. It was rolling 1-100. But then when modifying that chance by an NPC's statistics, it first multiplied that statistic by 0.01. It couldn't connect the fact that the rest of the program was already using whole number percentages (1-100) with a statistic modifying your chance to hit, and had decided to use 0-1.0 for that.
So be careful about how "smart" you think it is. It can churn out code. But it will also introduce bugs if you ask it to, and at least in this experiment would not assert itself in situations where my stated understanding of the program as written was wrong.
Source : https://youtu.be/h0962biiZa4?t=2486
void sort_string(char *str, int backwards) {
// Get the length of the string
int len = strlen(str);
// Use a bubble sort algorithm to sort the string
for (int i = 0; i < len; i++) {
for (int j = 0; j < len - i; j++) {
// If the current character is greater than the next character (or less than, if sorting backwards), swap them
if ((backwards && str[j] < str[i + 1]) || (!backwards && str[j] > str[i + 1])) {
char temp = str[j];
str[j] = str[j + 1];
str[j + 1] = temp;
}
}
}
}
and a detailed explanation of the fix and what the problems are.
The problem is that the working market might be flooded by this since way more people will be able to code ... Github Co-pilot is nowhere near this.
When an AI is able to train itself (like a human) is when developer jobs will be in trouble. Fortunately, every other profession will also be out of a job as well.
Until then, rote work will continue to be automated more effectively but no more.
But replacing computer operators is scalable. So I definitely expect most office people including low skilled developers to be replaced by AI in a 20 years. High-skilled developers might survive (and I'm not sure if I'd qualify) but obviously for most workers that's not very encouraging.
This is the part that is going to change the way people work in fundamental ways very quickly. It will turn good software developers into non-shitty project managers, project managers into ?, and mediocre developers into ?.
But once you get to even a mildly complex program - defined as a program with a handful of intertwining but otherwise independent parts, things seem to go wrong very quickly.
So, yeah, its good at solving leetcode hard. But it seemed very difficult to use when I tried building even a simple RPG combat system.
Rewrite the following in Rust:
size_t strlenNEON(const char *p)
{
const char *const top = p;
uint8x16_t c16 = vdupq_n_u8(0);
/* 16 byte alignment */
size_t n = reinterpret_cast<size_t>(p) & 15;
if (n > 0) {
uint8x16_t x = *(const uint8x16_t*)&p[-n];
uint8x16_t a = vceqq_u8(x, c16);
unsigned long mask = GetByteMask(a) << (16 + n);
if (mask) {
return __builtin_clz(mask);
}
p += 16 - n;
}
assert((reinterpret_cast<size_t>(p) & 15) == 0);
for (;;) {
uint8x16_t x = *(const uint8x16_t*)&p[0];
uint8x16_t a = vceqq_u8(x, c16);
if (isFound(a)) {
unsigned int mask = GetByteMask(a);
return p + __builtin_clz(mask) - top;
}
p += 16;
}
}
Then why does it seem to have some kind of theory of computation?
I gave it code to compute a Fibonacci sequence with nonsense names for functions and it immediately recognized the code (explainable by pattern matching).
I gave it a function and told it to modify the function so that it passes a test, and it did it without a problem.
I’ve had it build a function up by feeding it different test cases.
I had it explain kmeans to me, give me a first-pass solution, asked it how to optimize the first solution, had it do that for me …
I don’t know - it feels different. It also feels like it’s 1-2 years away from being the most useful co-worker on any given team and 5-10 years away from putting a number of us out of a job (or at least changing that job so much as to be something unrecognizable to us today)
I’m guilty of this with my dog. I can’t help it with her head tilts and deep stares! Her inner monologue is probably less sophisticated than I like to think it is.
This. I think we’re already seeing superhuman performance here, in the sense that many / most people can’t e.g. read a specification for a programming language and write simple programs in that language.
It is synthesizing.
You can ask it to update its answers. You can ask it to evaluate possible solutions, then write the code for one of the solutions. Then write the code to test the code. And so on.
It turns the coder into a curator, editor, and code reviewer.
Cool?
Doesn't sound half bad to me. I've enjoyed learning to code more than actual coding. I'd be happy in a paradigm where only code to learn, to practice, to express ourselves and leave the boring bits to the machines.
From one perspective, that’s what humans have always been doing. We create new things by welding other things together.
From another perspective, that’s not even a bad thing. Every word exists in the dictionary so what value are books? Unless the value is the structuring of all those things that already exist.
Human creation -> ChatGPT -> Human query -> Human interpretation
The last bit, the interpretation, is particularly important. Just like we're predisposed to seeing faces everywhere, we're predisposed to seeing meaning, and perhaps "intelligence", everywhere. In this case the meaning is very convincing since it comes from other humans, diced and sliced, but is merely presenting ourselves to ourselves in an interactive way, using our style of discourse.