I think the author is onto something – while AI might not be able to program per se, it can certainly be handed a code snippet and then use its huge corpus of Internet Learning™ to tell you things about it, code that looks like it, and ways (people on the Internet think) it might be solved better.

In that sense, it isn't replacing the programmer; it's replacing IDE autocomplete.

Co-pilot has been very useful the times I've used it. It's not perfect, but does cover a lot of boiler plate. It also makes it much easier to jump between languages.

It reminds me of arguments that it's not the computer that plays chess, but its programmers.

You can describe a GPT's response as a statistical average of responses on the internet (for quite a contrived definition of average), but at some point it will be easier to describe it as analyzing a snippet and forming an opinion (based on what people on the Internet think). Are we past that point? I'm not sure yet, but we are close.

Sure, maybe an argmax mixed with an RNG isn't really an average, but some would say it's quite a contrived definition of forming an opinion.

You can describe solution of any well defined problem with argmax, it doesn't make it trivial.

I think the author is operating in what I consider to be the sweet spot of current LLMs - where I can ask a question I don't know the answer to but can reliably spot bullshit (either because I know enough or through other means). I think there's a lot of value to be had when those conditions are met, and not just for coding.

Last night I sat down and tried using it to write an 8086 emulator. It got an simple emulation outline fairly quickly. But when it came to getting each of the instructions and interrupts correct. It fell very flat very quickly. What was interesting is that it made the exact same mistakes many early emulator writers make. You could then correct it and it would give it a shot at 'doing better'. I at one point got a bit bored with it and kept feeding it 'can you make that more compact/better'. It did an adequate job at that, eventually using templates and jump lists. It did not get very far using duffs device or dynrec but I am sure I could have guided it into doing that.

But probably for a majority of things like in emulation 'close enough is good enough'. That is an interesting finding of coding things up I think. This thing is also going to make seriously crazy amount of bugs we will be chasing for decades.

I had a similar experience asking it to write an API client. It wrote something very plausible but just concocted an endpoint that looked real but didn't exist.

Good luck having a ML model understand a 20 year old undocumented dataformat developed inhouse at a specific research lab to be used in their proprietary systems which are also undocumented and are a part of a spiderweb of interconnected systems at that lab (I have worked at this particular lab).

It will be a long time (if ever) until a ML model will be able to handle these situations (and I hazard a guess that most of the worlds active code is something akin to this).

As a supporting tool for the software engineers working there, sure. Just like a linter.

ChatGPT can do that right now. Just provide it example data in the input and it can extrapolate the format and interact with this abstraction (i.e. describe the format, write code to it, etc). LLMs don't just remix data it has seen before, they perform in-context learning. This means abstracting out patterns in the input and then leveraging it in generating output.

So I am not average and I am enamoured with gpt, simply because it presents high value to me now, more than some actual real humans. For me that’s enough revolutionary.

But on the other hand in new uncharted territory, it sometimes fails on the simplest shit: Asked it recently how to do one thing with enlighten (that I knew was possible with tqdm, but was almost sure not possible with enlighten). It just hallucinated up parameters to functions that didn't exist. Several rounds continued where it had that from, if different version. I asked it even for the reference where it meant it had that from.. and it referenced me fully confident a readthedocs url with tqdm and enlighten mixed, that didn't exist.. it is hilarious how it confidenlty can tell you one bullshit answer after the next.. dialogues always "hey are you really sure about xxx, did you look it up" "yeees, very certain, I did!" "But this doesn't exist" "Oooh, Im very sorry, you are correct and I am wrong, the next bullshit answer is: ..."

The history disappeared I hope I get it back once, but the dialogue til getting to "No, it may be not possible with this library" was amazing, I'm really scared for our futures building up on that and what will happen if everything from business presentations to lawyer letter exchanges will build up on that..(:

If you have, I don’t think you are like majority of devs (maybe not on HN, but in real life).

You sound lucky to have true, novel problems to solve each day. I’m with many here commenting that this is quite powerful stuff, especially when my day-to-day is writing simple CRUD apps, or transforming data from one format to another within an API, or configuring some new bit of infra or CI/CD.

I’d love to be challenged in some new way, and have access to truly fascinating problems that require novel solutions. But most enterprises aren’t really like that nor do that need that from majority of engineers.

They've been promising those since the early 2000's, but I'm still employed.

Some stuff has been automated by Zapier, IFTTT and the like, but most of the systems I'm integrating with don't have public APIs.

Obviously it's a cheeky example, but this would not be the first time in history a previously well-established career was upended in a (relatively) short amount of time. I'm a FAANG dev, I've got skin in the game too and I'm trying to be optimistic, but I can't help but be at least a little worried. From Wikipedia -

"In 1890 there were 13,800 companies in the United States in the business of building carriages pulled by horses. By 1920, only 90 such companies remained."

I don't think we'll be out of the job entirely, but I can definitely imagine the bar being raised and the compensation stagnating as we now have to justify the time and cost of our work compared to the near-free, near-instantaneous output of an LLM.

All that being said, if you've been working since the 2000s, you've got nearly a 20 year head-start on me, so perhaps it makes sense for me to be a bit more worried.

Yea, that's mostly why I get hired. Experience gives people a certain intuition on what kind of solutions work for which cases.

And when you've been working long enough, you don't (hopefully) feel the need to do cool bleeding edge shit at work, you just want the work code to ... work. You pick the simplest and most boring solution possible so you can clock out at 1600 and not think about services crashing because the newfangled thingamageek coded with the latest language du jour failed because of an edge case nobody has seen before.

Wow. The hype around "AI"s has entered the crypto stages. Just like crypto was the new car, the new internet etc., so is AI now.

Well, at least AI is useful in certain applications.

And horse carriages have some serious advantages, maybe even will make a comeback. :)

I feel completely differently about LLMs; I'd say we're closer to 2007 when the first iPhone was released. I believe LLMs will become a part of our day to day lives at a level that crypto never came close to.

It's same for me. I can learn how to tile my bathroom or repair my car, but i just don't feel like and am happy to pay someone else.

And you know what? We went from having actual no- or low-code tools (UI builders, forms in Access and FoxPro etc.) to zero no-code tools worth a damn [1]. There was a brief dream of Yahoo! Pipes in mid-to-late 2000s, but it's dead as well.

[1] Except some applications like Unreal Blueprints and similar node-based tools in audio and graphical software

Once you create a general purpose no-code option, it is so complicated and sprawling that the mental burden to understand it is just as great (if not greater) as just using plain old code again. Or conversely, it is so constraining (for the sake of "simplicity") that it can't do anything useful.

I think all of us here conflate simple with easy. It's simple in theory yes - you get some JSON from service X, maybe tweak service Y that talks to X and then feed it into some front end. In practice even very experienced engineers can take days writing or changing a simple end point or some front end because unclear requirements/bugs/micro service hell/unclear existing code/etc etc.

If it was that easy the pace and quality would have been much higher than what I'm seeing in tech companies.

I agree with GP - day 2 dev me outclasses it, which means it isn't replacing anyone.

I do agree that there is no boring crud; that’s why gpt or no code tools are not good for full solutions (yet), but it’ll get there I am sure.

Please elaborate.

And, if true, this would be a major news story that Vox or any number of major newspapers would love to write about - so have you approached the media about this? If not, why not?

I simply know firsthand (i'm old, i have manager, cto, ceo friends who I go golf and play squash with) that people in data entry and programming have been let go in the past weeks because 1 person could take over their work using the gtp/chatgpt api's and do their work faster with less errors. I am recommending the same in my company as a lot of my colleagues are doing nothing anymore as the skilled seniors are doing it themselves with gpt now as it's faster, less communication etc. We feed jira issues into gpt and it generates code; we review and refine or fix ourselves. It works much much faster and with better results. Most things most of us do all day is integrating ancient API's of partners and so mapping xml/soap/... api's to our json schema's. With chatgpt that's really fast and mostly painless; it even renames the properties that need to be changed to our enums properly. With humans this is a painful and slow process, especially with people who are fast and loose (broken education seems to made many of those graduate just by cheer production speed & volume instead of quality; gpt can do that better too...).

> so have you approached the media about this? If not, why not?

Why would I do that? Even anonymous, it doesn't seem to make much sense for me to do that. Anyway; that'll come soon enough as it will be common soon.

Also, what license is the code under when you get it back?

That's ok for your use case but risky for anything open source of that might be sold around the world.

This isn't software engineering work, this is 21st century data entry with some code. This is exactly the type of menial work that should be automated by AI.

If you have small self contained problems like map X -> Y then sure, ChatGPT will be sufficient. Where I disagree with you is calling these jobs "programming" jobs. These are the type of tasks that should've been written in a transform language like JOLT. This shouldn't even be code.

> With humans this is a painful and slow process, especially with people who are fast and loose (broken education seems to made many of those graduate just by cheer production speed & volume instead of quality; gpt can do that better too...).

Humans suck at repetitive menial tasks like this. It's not education's fault.

Who said it was? This is what most programmers do all day, that's the point. These people can be replaced now without writing specialised software for the case. It is programming, not software engineering and it is what people are doing all day long (and longer) who are called programmers / software engineers in most companies all over the world. You can disagree with it, but that doesn't change much.

So you are now putting the bar higher which is not fair; the fact is that people who have the title 'programmer' and even 'software engineer' are now readily replaced by AI. If you don't agree with the title; I don't either, but reality is what it is.

I would say basically the point is ; there are way way too many people being 'programmers' (but not limited to this field) who can be replaced; only a few % should remain as the rest does what you wouldn't call programming, but the rest of the world does. Search twitter for 'html programmer' and such. Millions and millions will never be programmers and your definition, but have a high paying job (for their country) working as a programmer.

At this point, this would only affect engineers who don't understand our subject area (biotech) and are relatively junior (in the sense that their output is not much better than a GPT4 output reviewed by a more senior dev).

But I still don’t know if I am thrilled by the prospect of being called in to make their monkeypatched, GPT generated project work.

You mean, like almost every outsourcing company pops over? So the type of code that infests companies who hired some sweatshop to do ‘some simple crud’? What’s the difference? Can you see the difference? Besides the gpt code will be far better commented as comments come for almost free with gpt while humans hate writing them.

I’ve seen GPT 3 and 4 hallucinate the most amazing commentary about their own output. Maybe we will get dependable, out of process, guidance at some point about how factual the model thinks it is on an output per output basis but until that point you should trust every LOC and comment exactly the same as code gifted to you by an adversary.

My modest 2¢

After validation.

After getting the changes to disk, documented, actually compiling, etc…

But the biggest problem is that transferring the nuance that is external to the code base is typically really tiresome and lengthy grunt work and again token cap.

The sad thing is that real luddites would go out and actively sabotage AI development because they think it's a real threat. Yet these people just makes bold and false claims in online forums and continue to move the goalposts once they're proven wrong. Sad. Pathetic. (and obviously, I don't mind being downvoted. Whatever! :)

The hard part here isn’t really the coding - but understanding the business logic and getting it out of the customer what they need.

Often the customer hasn’t even got a good enough grasp on their own business to be able to formulate what they need.

In the short term this ai tools will make the programming part easier but the planning part is still hard.

I also wanted to add about the myriad of incoming data formats that need to be processed and the myriad of data exports that one has to implement for most of those "boring" CRUD apps.

If one hasn't written code that includes comments like "Special case, California does it this way" or "Alberta needs an .xml export, not an .xsl one", with a link to a .pdf spec that points to somewhere on the internet, then he/she hasn't got to know what it really means to write a true and "boring" CRUD app.

I point chatgpt at the general problem and ask about the tech. Check it's answers, drill into specifics. As more questions, get it to write a prototype. (obviously with different parameters to prod- I don't trust open ai not to harvest from my interactions)- ok, now I have proof of concept, if it does the thing I want to do, then go to the engineers and say - hey I want to "X", here is RFC and rough code, .. any problems? if not, how long to push a localised version?

I guess you might call this scripting or prototyping not "real coding" but, damn it's useful not have to fill my head with python etc. Or bore/waste a guy earning 150k for a half day plus to get the basics, then never get my prototype.. because, priority, resources, etc

GPT has limited reasoning but given enough knowledge of the problem you can coerce it to do surprising things so long as you can relate it to something in else in the knowledge base. Given how big that knowledge base is, you can get lucky surprises where things just work if you fish around enough

I've worked on adtech, crypto, fintech, gamedev, startup founder, BigCo. Not once was programming something that was a time sink.

Makes me feel like GPT is marketing to the incompetent or something.

Can you please expound on that? Do you mean not hard as in comparison to something else?

No it doesn't, there is a lot of simple "gluing APIs together" to do in all of those. The hard part then is figuring out what you want to do, not getting the code to do it.

I did a startup for years and the backend was all boring Golang talking to Postgres. I am confident that if GPT had produced identical code it would've been worse. Because shit hit the fan many times due to misunderstandings or other things that cause bugs. Because I wrote the bugs, I was able to fix them. Making coding more of a blackbox and treating coding as menial labor would have definitely caused us to lose customers and miss important deadlines.

Maybe the next way to move the goalposts is "GPT will debug for you someday and be trained on the entire transitive closure of your dependencies."

That sort of thing could actually be useful as a sort of hint, but it isn't replacing devs any more than ripgrep did. To be honest, ripgrep through the source of my deps is already highly efficient and points me to things often.

In fact, all the complicated software anecdotes I could give were things that ChatGPT wouldn't even touch. In the realm of design and scaling and fault tolerance and other such things.

Absolutely. The common constant I can see in people who are really blown away by GPT's performance at [task] is that they are bad at [task].

Programmers who describe their job as copying from StackOverflow think it's great at coding. People who don't read fiction think it's great at writing fiction, and so on.

But it's certainly far better than humans who are not skilled at those tasks, and that is what I find to be very impressive. I just didn't realise that these models could be this good, and they're not even as good as they will be.

I guess if you were expecting something that's going to be as good as those who are skilled in a particular field, you'll be unimpressed -- but I wasn't even expecting mediocrity.

I'm saying that most people I see who are impressed by it for any given task are people who are not well-equipped to judge performance on that task.

Programming is generally not hard. Producing 5x - 10x as much programming output at the same quality is quite hard. And keeping the most interesting 10% for yourself while offloading the 90% that is basically just typing? That’s what people are excited about.

Yeah, I've gotten it to write some pretty decent contracts, but only because I have written said contracts and can ask it all the right questions/prod it into adding what I need.

Like how we've relied on fossil fuels that are a limited resources, to get further we have to go beyond such fuels. It seems like we're in the process of locking in our highest attainable level of text-or-image based advancement.

We mine the expertise in industries (programming, music production, graphic design, ...) and simultaneously cause the replacement of many future experts. Thus leading to a lack of expert output that we can mine?

Now, currently, the problem seems to be "we're putting people out of work" -- but that's progress. The real problem is that we are unevenly distributing the benefits.

Is GPT really like the old school NLP trigram stuff?

I'm sure if I kept probing on smaller and smaller details it would eventually fail, but I'd argue for _most_ people, on _most_ subjects, it performs incredibly well.

A real test of it's abilities would be synthesizing a NEW type of nuclear reactor.

1. Do you think that companies paying millions of dollars to ML researchers aren't already trying to make it good?

2. I think it will take a real revolution in AI/ML to do what people here are extrapolating into the future. That revolution will eventually come, but I doubt it'll be as quick as people think. Just think about the excitement people had about Siri 10+ years ago, or about Full Self Driving 5+ years ago. In my opinion, in 5-10 years from now GPT will be in the same place where Siri and Full Self Driving currently are. Eventually we will make the leap we're dreaming of, but that leap isn't happening yet.

The rate of increase in capabilities is also unpredictable, which is what is amazing & terrifying.

Overinflated hype about “where the puck is going” being wrong is…not a new phenomenon. And the non-tech media (traditional and social, and particularly a whole lot of the elite commentariat that spans both more than the “hard” news side of traditional media, though that is influenced too) perspective on this is driven quite disproportionately by the marketing message of the narrow set of people with the most financial stake in promoting the hype. Even the cautionary notes being sounded there are exactly the ones that are being used by the same people to promote narrow control.

The part that gets me is they aren't just aimlessly making posts, they're getting involved with webinars targeted at educators in their respective topics, speaking at regional events about the future with ChatGPT, etc.

One of them was a really excellent teacher but like this dude absolutely does not have any qualifications to be speaking on ChatGPT, and I hope it hasn't actually changed his teaching style too much because I'm having trouble imagining how ChatGPT could've fit in well with the way he used to teach.

Don't get me wrong, hype often surrounds something legitimately good, but I think ChatGPT is being taken way out of context in both the ML achievement it is and in what things the tool is actually useful for. I guess it is easy for a layperson to make mistaken assumptions about where the puck is going when they see what ChatGPT can do.

[1] "The story took place in 1929: Joseph Patrick Kennedy Sr., JFK's father, claimed that he knew it was time to get out of the stock market when he got investment tips from a shoeshine boy."

I grew up and using a pocket calculator was verboten. You just did not do it. You better learn and memorize all of that stuff. Spin on 10 years after me and all kids have them now. If you have the right app on your phone the thing will OCR the problem and auto solve it and show you the steps. ChatGPT and its ilk are here, now. How we learn and create things has dramatically changed in under a year. It is not clear how much though.

Teachers are trying to get ahold of what does it mean to teach if you can just ask some device to summarize something as complex as the interactions of the 6 major countries in WWII and what caused it. Then the thing thinks for 2 seconds and spits out a 1000 word essay on exactly that. Then depending on which one you use it will footnote it all and everything.

This style of learning is going to take some getting used to. This tool is in their classrooms right now. Right or wrong. It will be there. The teachers are going to have to figure out what this means to their class planning. Not 5 years from now, today.

These posts ooze hype bullshit, not nuanced talk about pros and cons and how it will affect education. ChatGPT should be thought of as a writing tool and perhaps an information source on very surface level topics. Trying to teach an advanced research course at a top school with ChatGPT heavily involved is a terrible idea on the other hand.

I have tested asking it about a number of specific topics in biology research/asking questions about particular papers, and it gives horrible answers the majority of the time. If someone submitted that as a paper to me I'd give it a bad grade because it is dumb, I wouldn't need to know if it were ChatGPT or not. I would be alarmed if my kid's teacher went from organically teaching how to dissect a scientific paper to suggesting that a major part of the curriculum can be replaced with talking to GPT.

I've seen articles about teachers taking the other extreme against ChatGPT too, but I haven't personally seen anything that was a realistic take on what LLMs can do. Maybe it boils down again to disagreement on "where the puck is going" but to me most of the hype is making ridiculous assumptions about what is imminent meanwhile ignoring the things worth discussing now.

Which sounds a lot like bubbles. The dot com crash didn't mean the internet was a bad idea or that it wasn't worth discussing at that time.

"where the puck is going" is something to consider and we will 100% get that wrong, but right now this thing is creating waves and making issues now.

Already happening. A student in Russia submitted his thesis written with the help of GPT, it was accepted, then rejected, then accepted: https://www.themoscowtimes.com/2023/02/02/russian-student-al...

In any event, he isn't concerned at all about students using it for assignments, it's the degree to which he seems to think it can be integrated in his current curriculum that alarms me. I think he is misunderstanding the capabilities and therefore pitching something that doesn't actually make sense.

The other teacher I didn't follow as closely, I just got a kick out of seeing she was now advertising some sort of webinar about ChatGPT filled with buzzwords too.

Also if a teacher is really serious about wanting to plan around what GPT can and can't do to the point they want to be a teaching authority on it, they should be consulting with people in the relevant domain. When I want to use a new tool in the lab I talk with colleagues that have a great deal of relevant experience before I start working it into experiments. I can't imagine starting to give advice to others on the matter when I don't actually know the current nuances.

And seriously, how many people were actually saying “WolframAlpha will destroy a huge amount of programming jobs”?

But it looks like an exponentially steep climb from here. indefinitely

Maybe superintelligence isn't possible for some reason, but the fact we exist should tell you that general intelligence is not magic.

Though I think that it's more likely that I'll become shaman who talks to the spirits. Managers hate computers and they'd prefer to pay someone to deal with them.

That rapid progression can be modeled with an exponential growth curve, like Moore’s law, even though nothing material can have a sustained exponential growth. Regardless of how the curve is modeled, the steepness is such that it serves as a step function in evolution.

With that clarified, there is first the question of “is the technological progression going to level off soon?” And I think the answer is no. The second question is, “how are we going to deal?” And the answer to that question can only be addressed once we answer the question: “what kind of world do we want to live in?”

People generally use the term to mean the point at which progress is automatic, unbounded, and instant. E.g. AIs that can produce smarter AIs themselves, in short order.

I’m not a singularity believer (singularitist?); from the perspective of 1000 years ago we’re already at the point of infinite progress in zero time. I think individuals, the culture, and the species adapt to accelerating progress, and I intuit, maybe wrongly, that Gödel‘s theorem means that technology will never become runaway and recursive.

But I think that’s what people mean, more than just a step/paradigm change like the internet.

Additionally, "the rate of increase in capabilities" is very much a false flag. Past performance (especially for second-order things like 'rate of improvement') is an absolute batshit insane metric for predicting future success.

https://russellinvestments.com/us/blog/past-performance-no-g...

The advancements have eaten low hanging fruit. Once all the low hanging fruit is gone, we'll all realize GPT will never be tall enough to reach the good stuff.

But the history of technology says that demand is recursive; the more tech produced, the more demand there is for producers.

There may be a time when we hit the old “the whole world only needs 5 computers”[0] limit, but I don’t think we’re anywhere close. AI is providing leverage to create net more programming; it is not replacing programmers in a zero sum game.

0. https://www.theguardian.com/technology/2008/feb/21/computing...

> In economics, the lump of labour fallacy is the misconception that there is a fixed amount of work—a lump of labour—to be done within an economy which can be distributed to create more or fewer jobs.

This is a very bold claim IMO.

Modeling/understanding interactions in a complex system of potential black boxes is much, much more computationally difficult problem that source code to source code operations.

Last couple of months, I've been using chatGPT to write a lot of features and functions. I don't think it has made me a better coder, but it has made me massively more productive. Things like scraping data from a URL - something I would have had to sit through an hour long tutorial to learn - is accessible with a single query.

I also think that the code quality has improved over the last few iterations. It makes fewer mistakes now.

Exciting times

This is another one of those rebellions, non-programers hoping to avoid reading the book and closing it for good, while keeping the awesome around. The code-bases we will see, were the commits are basically chatgpt tags and tasks for each document.

Well it is revolutionary. And it isn't just where it is today, but how fast these models are improving - with no saturation in ability evident at this time.

On the other hand, I am not sure anyone is saying programmers are infantile - although poorly written software is as at least as prevalent as poorly compensated or managed software development positions.

  > I really am in awe of how much work people seem willing to do to justify this as revolutionary and programmers as infantile, and also why they do that. It’s fascinating.

But fret not, programming is not the first industry that has been automated into a shell of itself. Yes, the industry is going to shrink massively, but this is what new skills are for. Just as farmers had to learn industrial jobs and then miners and autoworkers had to "learn to code", most programmers will have to learn to do something else. Humans are resilient and will adapt.

And there will still be jobs in development for the most talented and in niche areas, but when the largest tech companies can layoff hundreds of thousands of employees without skipping a beat that should tell you all you need to know about the value of most "programming" jobs.

The transition from agrarian to industrial societies was extremely painful and arguably it was centuries before the people affected were better off.

> Humans are resilient and will adapt.

Based on recent events I think it's more likely people will elect extremist politicians who promise quick fixes while blaming a convenient out-group for everything.

You meant "expand massively" i think. Did all the programmers manually making machine code get fired and the job of programmer disappear when compilers were invented and totally replaced these jobs? No, it just changed to use the new tool.

There wont be any unprecedented mass layoffs, despite what the jaded STEM-hating twitter crowd wants. Companies will simply make their programmers use these tools to increase the amount of produce per employee, and make software that would have been otherwise financially impossible to be made. Because the competition will do so too to get ahead.

  > You meant "expand massively" i think. 

> There wont be any unprecedented mass layoffs

Baghdad Bob strikes again.

What i think will lead to mass layoffs is the current recession rather.

  > this is nothing new or unprecedented

Like what? Seriously, which cognitive occupation is safe then? I think if one wants to stop competing with the machines (who appear very close to becoming superior to humans by what you are saying), it's some kind of child care / social work job. We still don't want robots in those (for now. Eventually they may do those better than us as well).

Best case, that will have a whole lot more humans using their right brain halves on things like defining the problem. I like the thought of that, it's more pleasant work. But a lot of intelligent people define their intelligence by how well their left brain half works and uncomfortable with how good Chatgpt is at those tasks. I think you're pointing out there's more to programming than left-brain activities, and I think you're right that silicon will never eclipse carbon at those challenges, but I think a lot of people are also feeling threatened by the fact that chatgpt is getting better and better at the thing they used to be better than all humans at.

We might be able to state that GPT will easily trim away all the average workloads for both the left and right. It can perform "creative writing" or draw pictures to an average or even beyond average extent, but it continues to currently struggle to hit the exceptional examples that humanity are capable of.

I think 'left brain' work also has a lot more predictability (knowing you can churn out ~X widgets/hr) so having only uncertain 'right brain' work can be uncomfortable for people to build their livelihoods upon.

That being said. 'right brain' work is certainly more fulfilling for me.

Absolutely. Not everyone can be consistently creative, and without a large enough financial buffer, it's a recipe for disaster.

* ChatGPT is revolutionary - honestly, it's genuinely impressive how much of a leap ChatGPT is compared to the attempts that came before it.

* Programmers write a lot of simple code that has been written before - there are genuinely tons of cases of "write a web endpoint that takes an ID, looks it up in a database table, pulls an object through an ORM, and returns a JSON serialization of it." Most programmers? Doubt it. But tons of programmers write CRUD stuff and tons of IT admins do light scripting, and a lot of it is repeated code.

Could ChatGPT do my job? Not even close. But it's still really impressive to me.

What they failed to predict was that some people wouldn't try to automate them like-for-like. Instead they would reconfigure their entire approach to fit with the specific advantages and limitations of the machinery. And this new approach might even be qualitatively worse in various ways, but not so much as to overwhelm the economic advantages that provided by the things machines were good at.

AI likely isn't going to slot into a developer-shaped hole in a software team. But it's possible we'll see new organisation approaches, companies, and development paradigms that say: How far can you get if you put prompt-generated code at the heart of the workflow and make everything else subservient to it. I'm not sure, right now, that that approach is feasible, but I'm not sure it won't be in a year or two.

Their competitive advantage in their field then becomes the range of their tests (borne through experience), efficiency in running their pipeline of testing and ability to generate effective prompts.

We had to do that for the industrial approach and it wasn't a simple, fast or intuitive process.

OK, challenge accepted - will go down your suggested route - thanks :)

We won't have understanding of our code, similar to how we don't understand the machine language being generated by our compilers now.

We will be using our intuition about GPT to bring into being fully designed and integrated systems with 10 paragraphs at the prompt.

Which could in the end greatly increase the influence of a programmer in a given organization. Our role will be a softer, almost cyborgian one.

But this will indeed require the destruction of all that came before it. Questions like "but does it work with this 3rd party API, or this platform?" must become irrelevant for this future to happen.

A bit similar to how the web destroyed mainframe, perhaps, by first creating its own compelling world, then making the mountain come to it.

The more black box programming becomes the more dumb human programmer gets. There will be stagnation. There won’t be any new “design patterns”.

1. The whole bullshitting thing. If I bullshitted even 1% as much as GPT I’d be fired within a few months.

2. Precision. Ask it to count, or do simple math. It seems to really struggle here, and despite us feeding it nearly all of humanity’s text, it isn’t getting better. Programming is extremely precise in a way painting isn’t. GPT is a painter, not a programmer.

Final note:

9 years ago I worked on an ML startup with a few people, some of which went on and now are at OpenAI. We saw the absolutely incredible progress ML was making and had many discussion with the brightest in the field.

I was much like you, I was convinced that ML would he automating basically everything within 5 years. I mean the rate of progress felt super-exponential and there was hard proof of amazing results. Especially things like radiology for example which seemed to almost already be there.

Turned out to be dramatically wrong. My 5 year predications were laughable. It did well on some things we didn’t expect (fuzzy stuff like language and drawing) but much worse in the precise areas. It still is struggling with precision, and radiology in the most straightforward cases.

I think my calibration on this is much better now. I realize it has limits, where they are defined, and I see now it doesn’t scale in some ways. It doesn’t produce novelties in the way humans do while also being a lot less conservative/precise. It may turn out that solving both of those things at once is the extremely difficult last 10% that is actually 90% of the work (and they feel like polarities that may train against each other - be very precise, also don’t lie, but still be creative - have you noticed the more you push GPT the more “distracted” it gets? I see limits in many ways as it’s scaled that don’t look good for this).

Look at Self Driving Cars. 7 years ago I predicted they’d be here within a year or two. Had to be. They still aren’t! Why? The last 10% is really, really hard. It’s also critical to success. We really seem to have trouble reconciling these two, both underestimating the difficulty, and the need.

Stuff like your “a year or two” note is a big red flag for me here, especially when the claim is so broad. Yes, it will integrate with things in a way that avoids a lot of complexity developers deal with, but needless to say I’d love to see a concrete bet for “in two years it will have (insert exact large effect)” so I can profit off the hype.

However, the next challenge is that the machine itself is now an "artisan" device.

I'm sure the first bowl printing machine ( https://youtu.be/bD2DNSt8Wb4 ) was purely artisan... but now you can buy them on Alibaba for a few thousand dollars ( https://www.alibaba.com/product-detail/Printing-Machine-Cera... )

I am sure there is a (bowl printing machine) machine out there.

But if you say "I want a bowl printing machine that can do gradient colors" that first one (and probably the first few until it gets refined) are all going to be artisanal manufacturing processes again.

This all boils down to that at some point in the process, there will be new and novel challenges to overcome. They're moving further up the production chain, but there is an artisan process at the end of it.

The design of a new car has changed over time so that it is a lot more automated now than it was back then ( https://youtu.be/xatHPihJCpM ) but you're not going to get an AI to go from "create a new car design" to actually verifying that it works and is right.

There will always be an artisan making the first version of anything.

Until we reach the bootstrap point (the singularity?), i.e. when the GPT-making machine is GPT itself. Or maybe we're still one level behind, and the GPT-making machine will generate the GPT-making machine, as well as all the other machines that will generate everything else.

For one, it made horrible, glaring mistakes (like defining extern functions which dont exist, using functions which are specific to a platform im not using, etc.), stuff beginners would do.

It also decided to sneak in little issues, such as off-by-one errors (calling write() with a buffer and a size that is off by one in a place where its very hard to tell), missing edge cases (such as writing a C++ concept which worked, but actually did everything in slightly the wrong way to actually ensure the concept was requiring exactly what I asked).

Even when asked to correct these mistakes, it often struggled, made me read paragraph after paragraph of "im sorry, ive been such a bad little machine" garbage, and didnt even correct the issue (or, in some cases, introduced new bugs).

Im utterly unimpressed by this. GPT is great for a lot of things, but not writing code better than I would, in the same time.

The time it took me to massage it to solve a nontrivial problem (write hello world with just syscalls) was way longer than reading the manual and writing it myself (and has less bugs).

Not everyone unfazed by these articles is simply in denial. I feel sorry for people who write copy paste code and find that ChatGPT or Clippy from 2000 can relace them, but not everyone writes trivial code.

On the one hand we don't even have a roadmap toward reliable AI.

On the other, if we ever plug an LLM into something that has memory, acquires experiences, does experiments, observes the outcome and adjusts its worldview in response, consciousness might fall out of that. And writing good code might not even require consciousness.

- as a function of an independent human soul

- as the fundamental substrate on which the rest of the universe is built

- as a byproduct/secondary phenomenon of physical processes

In the latter two cases I believe that the question of whether GPT is conscious is immaterial. In either case it is functioning in the same medium we all are when we talk, think, write. In the first case it is not, and the question is thornier.

One can debate whether either those is necessarily a consequence of consciousness, but nonetheless those kinds of qualities are what people are aiming at when they wonder about conscious AI.

What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.

I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.

My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461

presumably they've already fed it all the code in the world (including a load they probably aren't licensed to)

The problem I've seen is that, maybe like the author has been writing, it's making sh*t up. That's not untrue, sometimes I didn't give it all dependent classes and it tried to think sometimes correctly, sometimes incorrectly what those were (such as method signatures, instance members, etc.) I wish it would have asked me some details rather than trying to figure things out. The guys at OpenAI have still a lot to do, but the current status is very impressive

it doesn't understand anything, it doesn't make deductions

it's a probability model, and I understand how it's implemented perfectly well, thank you

The point being, you can choose to laden the word “understand” with the meaning of human-like thinking, in which case humans will always be superior by definition. Or you can choose a “many ways to Rome” definition of understanding that is purely focused on results.

Large language models understand language in their own way. Currently their results are inferior to humans’ but one day the results may be superior.

This is data at a scale OpenAI did not have access to before.

What I find special about the TNGS is the Darwin series of automata created at the Neurosciences Institute by Dr. Edelman and his colleagues in the 1990's and 2000's. These machines perform in the real world, not in a restricted simulated world, and display convincing physical behavior indicative of higher psychological functions necessary for consciousness, such as perceptual categorization, memory, and learning. They are based on realistic models of the parts of the biological brain that the theory claims subserve these functions. The extended TNGS allows for the emergence of consciousness based only on further evolutionary development of the brain areas responsible for these functions, in a parsimonious way. No other research I've encountered is anywhere near as convincing.

I post because on almost every video and article about the brain and consciousness that I encounter, the attitude seems to be that we still know next to nothing about how the brain and consciousness work; that there's lots of data but no unifying theory. I believe the extended TNGS is that theory. My motivation is to keep that theory in front of the public. And obviously, I consider it the route to a truly conscious machine, primary and higher-order.

My advice to people who want to create a conscious machine is to seriously ground themselves in the extended TNGS and the Darwin automata first, and proceed from there, by applying to Jeff Krichmar's lab at UC Irvine, possibly. Dr. Edelman's roadmap to a conscious machine is at https://arxiv.org/abs/2105.10461

There are so many non-CRUD complex disciplines involving programming such as signal processing, robotics, control theory, scientific computation to name a few, the current version, at least, of GPT is not even close to being a good supplement, let alone a substitute.

But then I remember I'm on HN where the technical pinnacle of programming is Backend and DevOps.

The bot created the whole unit test involving the creation of data with test fields, then queried the output results and put some asserts. That's more than 100 lines of code which were written by GPT4. A (good) Salesforce developer would need a good 30 minutes to write those, and the result would not have been better.

Again, I also have some counter examples were it made some mistakes, but this is really shocking how... a program... figured all this out.

I think It'll help with some tasks, which is always good to take. After all, people tweak their vim settings because they feel it makes them more productive.

It seems on par with the Industrial Revolution, at least. Which, easy to forget, was a huge deal that totally changed society.

https://youtube.com/watch?v=MNtjSMvcjzs&t=211s

- a lot of programmers, including experienced ones, are absolutely infantile and they only have a job because there is a big shortage of programmers; not all of them get better with experience... hence a significant part of software development is dealing with problematic programmers and problems created by them.

- GPT is not that great a programmer but a great thing about it is that it is not a human... and one can get thousands of instances of them for the price of one human. You only need one of those instances to produce usable code.

- there have been many changes throughout the years which have definitely replaced a lot of programmers: library distribution services (pypi, npmjs), better software development tools and practices, SaaS delivery model, better programming languages etc.; so far, because the market need for programmers has continued to increase, most programmers continue to have jobs; this won't last forever.