Hahaha, if you want nit-picking, all the language tasks chatGPT is good at are strictly human tasks. Not general tasks. Human tasks are all related to keeping humans alive and making more of us, they don't span the whole spectrum of possible tasks where intelligence could exist.
Of course inside language tasks it is as general as can be, yet still needs to be placed inside a more complex system with tools to improve accuracy, LLM alone is like brain alone - not that great at everything.
This extreme difficulty in discerning what it hallucinates and what is "true" is what it's most obvious problem is. I guess it can be fixed somehow but right now it has to be heavily fact-checked manually.
It does this for computing questions as well, but there is some selection bias so people tend to post the success-stories and not the fails. However it's less dangerous if it's in computing as you'll notice it immediately so maybe require less manual labour to keep it in check.
I have a feeling they had access to a lot of code on GH, who knows how much code they actually accessed. The conspiracy theorist in me wonders if MS just didn’t provide access to public and private code to train on, they wouldn’t have even told Open AI, just said, “here’s some nice data”, it’s all secret and we can’t see the models inputs so I’ll leave it at that.
So yeah I feel your enthusiasm but if you think about it a little more, or maybe not so hard to imagine what you saw being actually rather simple ? Every time I write code I feel kind of depressed because I know almost certainly someone has already written the same thing and that it’s in GitHub or somewhere else. ChatGPT just takes the knowing where to find it and gives it to you directly.
Nothing is magic, right ?
Regardless of the truthiness of your assertion, that general description of work is a majority of my profession. Novel contribution is a very small percentage of the work I do. I'm looking forward to that shifting significantly in the coming years.
It doesn't matter if the "POSIX compliant C" version of dirbuster was in it's training set. Or if my cloudflare wrangler JS API <-> Neon.tech SQL database implementation was. (Copyright arguments withstanding for sure.)
What matters is that it did 100% of the work of generating the code. All I did was guide it, code review it, and ship it. It felt like I had a team of junior engineers working in the trenches on the problem for me and I was getting to have fun playing at a level up the stack. It can generate 100LoC faster than I can type it, and I can code review it as it generates it. I can fix the code up in the editor and paste it back in with the next prompt for another round of work.
This is with the current model.
Unlike GP's assertion that people are falling for marketing, I actually don't know what OpenAI's marketing is. 100% of my belief in GPT-4 comes from it doing my real day-to-day tasks along side me. I don't need a crystal ball to predict what AI is going to do, I'm already using it as a partner in writing my code. It's already well enough adept at my profession to pair-program through problems with me.
This week I had a particularly hairy problem I needed to solve, and I was already behind schedule on the project. ChatGPT and I knocked it out in ~30 minutes. I suspect, if I were alone, it would have taken me several hours. The velocity boost came from: ChatGPT correctly identifying it as a binpacking problem (which I had missed), listing out several algorithms for approaching binpacking, and giving me an initial (incorrect) first implementation. I was able to go back and forth on that response and get the rough constraints figured out for a solution, ask it to generate that solution, and then clean it up in my editor.
The flow I'm using today is already a 100% increase in effective "workforce" for a software organization with a $20 per month per employee subscription to ChatGPT. Everyone gets a real-time, always available, pair programmer.
Okay, now try being a scientist in a scientific field that isn't basic coding.
It's not people laughing at pretences, it's people who know even basic facts about their field literally looking at the output today and finding it deeply, fundamentally incorrect.
I wonder what your personal success rate would be if we did a Turing test with the “people” who “know basic facts about their field.” If they sat at a computer and asked you all these questions, would you get them right? Or would you end up in slide decks being held up as a reason why misnome doesn’t qualify as AGI?
I find comfort in knowing that it can’t “do science.” There is a massive amount of stuff it can do. I’m hopeful there will be stuff left for humans.
Maybe we’ll all be scientists in 10 years and I won’t have to waste my life on all this “basic coding” stuff.
I asked GPT-4 to give me a POSIX compliant C port of dirbuster. It spit one out with instructions for compiling it.
I asked it to make it more aggressive at scanning and it updated it to be multi-threaded.
I asked it for a word list, and it gave me the git command to clone one from GitHub and the command to compile the program and run the output with the word list.
I then told it that the HTTP service I was scanning always returned 200 status=ok instead of a 404 and asked it for a patch file. It generated that and gave me the instructions for applying it to the program.
There was a bug I had to fix: word lists aren’t prefixed with /. Other than that one character fix, GPT-4 wrote a C program that used an open source word list to scan the HTTP service running on the television in my living room for routes, and found the /pong route.
This week it’s written 100% of the API code that takes a CRUD based REST API and maps it to and from SQL queries for me on a cloudflare worker. I give it the method signature and the problem statement, it gives me the code, and I copy and paste.
If you’re laughing this thing off as generating unhelpful nonsense you’re going to get blind sided in the next few years as GPT gets wired into the workflows at every layer of your stack.
> pushing the bottom out of the labor market and actively making the lives of the poorest worse in order to line their own pockets.
I’m in a BNI group and a majority of these blue collar workers have very little to worry about with GPT right now. Until Boston Dynamics gets its stuff together and the robots can do drywalling and plumbing, I’m not sure I agree with your take. This isn’t coming for the “poorest” among us. This is coming for the middle class. From brand consultants and accountants to software engineers and advertisers.
Software engineers with GPT are about to replace software engineers without GPT. Accountants with GPT are about to replace accountants without GPT.
> Literally every single argument I've seen towards this being AGI is
Here is one: it can simultaneously pass the bar exam, port dirbuster to POSIX compliant C, give me a list of competing brands for conducting a market analysis, get into deep philosophical debates, and help me file my taxes.
It can do all of this simultaneously. I can't find a human capable of the simultaneous breadth and depth of intelligence that ChatGPT exhibits. You can find someone in the upper 90th percentile of any profession and show that they can out compete GPT4. But you can't take that same person and ask them to out compete someone in the bottom 50th percentile of 4 other fields with much success.
Artificial = machine, check. Intelligence = exhibits Nth percentile intelligence in a single field, check General = exhibits Nth percentile intelligence in more than one field, check
This is AGI, now we are nit-picking. It's here.