There's actually two things going on when I'm coding at work:

1) I'm writing the business code my company needs to enable a feature/fix a bug/etc. 2) I'm getting better as a programmer and as someone who understands our system, so that #1 happens faster and is more reliable next time.

Using AI codegen can (arguably, the jury is still out on this if we include total costs, not just the costs of this one PR) help with #1. But it _appreciably bad_ at #2.

In fact, the closest parallel for #2 I can think of us plagiarism in an academic setting. You didn't do the work, which means you didn't actually learn the material, which means this isn't actually progress (assuming we continue to value #2, and, again, arguably #1 as well), it is just a problem in disguise.

Plagiarism is a great analogy. It's great because what we call plagiarism in academic setting, in the real world work we call collaboration, cooperation, standing on the shoulders of giants, not reinventing the wheel, getting shit done, and such.

So the question really is how much we value #2? Or, which aspects of it we value, because I see at least two:

A) "I'm getting better as a programmer"

B) "I'm getting better as a someone who understands our system"

As much as I hate it, the brutal truth is, you and me are not being paid for A). The business doesn't care, and they only get so much value out of it anyway. As for B), it's tricky to say whether and when we should care - most software is throwaway, and the only thing that happens faster than that is programmers working on it changing jobs. Long-term, B) has very little value; short-term, it might benefit both business and the programmer (the latter by virtue of making the job more pleasant).

I think the jury is still out on how LLMs affect A). I feel that it's not making me dumber as a programmer, but I'm from the cohort of people with more than a decade of programming experience before even touching a language model, so I have a different style of working with LLMs than people who started using them with less experience, or people who never learned to code without them.

Any reasonably smart company will invest in its employees. They should absolutely be training you to be better programmers while you are on the job. Only shit companies would refuse to do so.

That's true, but by the same token, it's also true the world is mostly made of shit companies.

Companies are just following local gradient they perceive. Some can afford to train people - mostly ones without much competition (and/or big enough to have a buffer against competitors and market fluctuations). Most can't (or at least think they can't), and so don't.

There's a saying attributed to the late founder of an IT corporation in my country - "any specialist can be replaced by a finite amount of students". This may not be true in a useful fashion[0], but the sentiment feels accurate. The market seems to be showing that for most of the industry, fresh junior coders are at the equilibrium: skilled enough to do the job[1], plentiful enough to be cheap to hire - cheap enough that it makes more sense to push seniors away from coding and towards management (er, "mentoring new juniors").

In short, for the past decade or more, the market was strongly suggesting that it's cheaper to perpetually hire and replace juniors than to train up and retain expertise.

--

[0] - However, with LLMs getting better than students, this has an interesting and more worrying corollary: "any specialist can be replaced by a finite amount of LLM spend".

[1] - However shitty. But, as I said, most software is throwaway. Self-reinforcing loop? Probably. But it is what it is.

Sometimes knowledge and experience aren't additive: if none of the students had a certain experience/knows a certain fact, the sum of the students will still not have that experience or not know the fact.

This is especially true now that senior employees have gotten more demanding about wanting to see theit children, which didn't happen at scale in the past.

Hence, regulation.

I rather see the problem in US culture: in many other countries switching the company every few years is considered to be a sign of low loyalty to the company, thus a red flag in a job application.

Then they both chuckle and say they’ll hire some other schmuck, it doesn’t matter bc they already got series A funding.

The latter represents a mindset that's prevalent at a large portion of companies. Most companies aren't FANNGS or AAA Game Studios (w/e) looking for the best of the best, most companies are outsourcing a large portion of work and/or are speeding to the bottom of the quality race. Many aren't even in any position to judge competence, nurture it, or reward it.

They just want "5 years experience" in whatever Cloud crap and Java thing.

That is absolutely not true. A much closer analogue of industry collaboration in the academic setting would be cross-university programs and peer review. The actual analogue of plagiarism is taking work of another and regurgitating it with small or no changes at all while not providing any sources. I'm sure you see what this sounds more akin to.

That's only webdev and games.

In academia, if a result is quoted, this is called called "scholarship". If, on the other hand, you simply copy-paste something without giving a proper reference, it's "plagiarism".

This has been going on for decades. It is called outsourcing development. We've just previously passed the work to people in countries with lower wages and conditions, now people are increasingly passing the work to a predictive text engine (either directly, or because that is what the people they are outsourcing to are doing).

Personally I'm avoiding the “AI” stuff¹, partly because I don't agree with how most of the models were trained, partly because I don't want to be beholden to an extra 3rd party commercial entity to be able to do my job, but mainly because I tinker with tech because I enjoy it. Same reason I've always avoided management: I want to do things, not tell others to do things. If it gets to the point that I can't work in the industry without getting someone/something else to do the job for me, I'll reconsider management or go stack shelves!

--------

[1] It is irritating that we call it AI now because the term ML became unfashionable. Everyone seems to think we've magically jumped from ML into _real_ AI which we are having to make sure we call something else (AGI usually) to differentiate reasoning from glorified predictive text.

But we have:

  - LLMs pass the Turing test more convincingly than anything before
  - LLMs are vastly more popular than any AI/ML methods before it
  - "But they don't reason!" -- we're advancing chain of thought

LLMs are also ML (using statistical methods)

I know this sounds like moving the goalposts, but maybe all this does is show that the Turing Test was not sufficient?

> LLMs are vastly more popular than any AI/ML methods before it

Are we back in high school? Popularity contests?

The implications of passing the Turing Test are profound. Tricking a user with a 5 minute conversation isn't the same. The examiner is allowed to be an expert questionner.

For one thing, software engineering would quickly be taken over as a client can just chat with an AI to get the code. That is far from happening, even with all the recent LLM advances.

Similarly for literature, science etc. Currently, you can detect a difference between a competent human & machine in all these fields just by text chat. Another way of saying this is that the Turing Test is AI-complete and is a test for AGI.

https://courses.cs.umbc.edu/471/papers/turing.pdf

> "The Imitation Game"

> "Can machines think?"

We may need to redefine what these words mean.

Whether machines can think, and whether we believe so, are distinct.

If we're just a kind of machine, we can just say that LLMs have not surpassed us.

While LLMs still give away obvious signatures (their flaws are LLM-y and not humanlike), they play the imitation game better than any other machines before them. I lived long enough and played enough with simpler chatbots to believe that the simple text-based imitation game that Turing suggested will be won by computers before machines achieve reasoning ability that surpass the smartest humans.

> [...] the Turing Test is AI-complete and is a test for AGI.

Since Turing never used the term AGI, this is an interpretation.

Since we cannot determine whether a machine can think by comparing their anatomy to ours and drawing a conclusion that whatever we do, they must do the same, we are bound to compare their ability to imitate us. Which means we can't ultimately know, we can just feel ourselves convinced.

So there's no objective point at which to place the goalpost.

So constantly moving it is an expression of acclimatisation.

I'm impressed by LLMs, in spite of the hype and the homework cheating.

If anything, LLMs ability to imitate human work how far from thinking a lot of human behavior is.

That is not true. Even our cells, with features that look a lot like machinery such as a proton pump, is on the whole several orders of magnitude more complex than any machine. Even a single human cell is more like an ecosystem than a machine. Let alone entire humans or even just the human brain. Consider that both cells and humans are capable of reproduction.

> If anything, the ability of LLMs to imitate human labor

Within extremely narrow confines and quite often going over into just conjuring up nonsense.

Good point. Not to mention the massive reliance on organisms like mitochondria and bacteria that don’t even share the host’s DNA.

> [imitate human labor] within extremely narrow confines and quite often going over into just conjuring up nonsense

Much like human labor.

So the intelligent, reflective and thoroughly iterated work is hardly replicated at all, and the poorly imitated, easily repeatable coursework and boring paper sludgework excellently so.

So we don’t just get to criticise LLMs for not actually being intelligent. We similarly get to criticise humans for not being so when we might think we are, either.

So? Does your car stop being a machine just because it's a complex systems of moving parts, many of which are dynamically balanced through feedback loops, which involve components sourced from different vendors, and substances that are not part of the original manufacturing data sheet?

Exactly what insight does this give us? I feel this is trying to contrast a single machine "unit" against a complex system, while also sneakily committing a naturalistic fallacy by using "machine" vs. "ecosystem" to imply "machine" vs. "life" in the magical sense (i.e. as if life was something beyond a physical process).

Why? I'd say it's been self-evidently true for at least three decades now.

> Even our cells, with features that look a lot like machinery such as a proton pump, is on the whole several orders of magnitude more complex than any machine.

How many is several? I think that, at the level of a proton pump, individual components comes close to the order of complexity humanity deals with in man-made machines.

Also, at this level, things really look like machines, act like machines, quack like machines - there's no reason to not call them machines, given they obviously are. It's naturally originating molecular nanotech.

> Even a single human cell is more like an ecosystem than a machine.

Certainly. But then, an ecosystem is defined as a system made of bunch of varied stuff interacting with each other, finding balance through a set of feedback loops. An ecosystem of machines is still an ecosystem, and is arguably a machine in itself, too - after all, the term "machine" also applies to self-balancing / feedback driven systems, ever since we invented control theory and formalism to describe feedback loops.

> Let alone entire humans or even just the human brain.

Complex machines don't stop being machines when you keep adding moving parts to increase complexity. Or, at which point between a protein pump and a human being you believe the assemblage of molecular nanotech stops being a machine?

> Consider that both cells and humans are capable of reproduction.

What's that supposed to tell us? Human reproduction involves cell reproduction.

Further, we can even disallow decisions based on the human/machine signature. The computers don't need to imitate human frailty and examiners decide based on the quality of the output, not whether it feels human or machine. I think that this is also why one of the stipulations is that the human/machine is not physically visible.

But even then, this goal is far away. We are not asking just about simple short chat imitation games but whether a computer can operationally do everything humans are capable of in the textual medium. This often requires experts.

The chat game version can be solved. But, the foundational goal is far away.

So it's quite neat that we've managed to beat the test by brute-forcing the right NN architecture on a goal function that literally is just "give the same response as a thinking human", in full generality of that phrase.

> But even then, this goal is far away. We are not asking just about simple short chat imitation games but whether a computer can operationally do everything humans are capable of in the textual medium. This often requires experts.

That is moving the goalposts. Or just another game entirely - which is fine on its own, as a further milestone.

Just like the original test, there are 3 participants in a test: an examiner, a human, and an AI.

The examiner may ask any question or series of questions to the human &/or AI, and receive responses. This is done via text or other means which prevent the examiner from directly seeing the body of responder.

Both AI and human are to attempt to convince the examiner that they're the human.

Unlike the original test, it then needs to repeat with different examiners.

If there exists an examiner which can distinguish an AI from a human (for all humans) in a Turing test with probability non-negligibly different from 50%, then the AI fails this meta-Turing Test.

Just because an AI can fool some people doesn't mean it's a human-equivalent intelligence.

Existence is too strong a criterion. Even correcting for random chance, if there's 1, 10, 100, even 1 000 000 examiners who could distinguish an AI from a human with high probability, that steal means the AI passed the test in practice, because even a million examiners is still a rounding error compared to the human population.

Being able to converse (or convince someone the conversation is natural) is not the same as being able to perform a technical task well. The same applies to humans as well, of course, which perhaps brings us back to the outsourcing comparison — how often does that work out great or not?

> LLMs are vastly more popular than any AI/ML methods before it

Popular is not what I'm looking for. I've seen the decisions the general public make elsewhere, I'm not letting them chose how I do my job :)

> "But they don't reason!" -- we're advancing chain of thought

Call me back when we've advanced chain of thought much more than is currently apparent, and again: for being correct in technical matters, not conversation.

I have done some romhacks, so I have seen what compilers have done to assembly quality and readability. When I hear programmers complain that having to debug AI written code is harder than just writing it yourself, that's probably exactly how assembly coders felt when they saw what compilers produce.

One can regret the loss of elegance and beauty while accepting the economic inevitability.

I think there is a difference here for the future of humanity that has never happened before in our tool making history.

There are a lot of innovations that helped us not do heavy thinking ourselves. Think calculators. We will just move to a higher level of magnitud problem to solve, software development is a means to an end, instead of thinking hard about coding we should be thinking hard about the problem being solved. That will be the future of the craft.

It's one of those problems that seems easy, but isn't. The issue seems to come out when we let an aid for process replace gaining the knowledge behind the process. You at least need to know what you don't know so you can develop an intuition for when outputs don't (or might not) make sense.

https://chadnauseam.com/coding/random/calculator-app

(recently: https://news.ycombinator.com/item?id=43066953)

It's especially silly because one thing calculators are known for is being inconsistent about order of operations between designs.

> It’s really about basic human personal insecurity, and we all have that, to some degree. Getting around it, is a big part of growing up (...)

I believe he's right.

It makes me think back to my teenage years, when I first learned to program because I wanted to make games. Within the amateur gamedev community, we had this habit of sneering at "clickers" - Klik&Play & other kinds of software we'd today call "low-code", that let you make games with very little code (almost entirely game logic, and most of it "clicked out" in GUI), and near-zero effort on the incidental aspects like graphics, audio, asset management, etc. We were all making (or pretending to make) games within scope of those "clickers", but using such tools felt like cheating compared to doing it The Right Way, slinging C++ through blood, sweat and tears.

It took me over a decade to finally realize how stupid that perspective was. Sure, I've learned a lot; a good chunk of my career skills date back to those years. However, whatever technical arguments we levied against "clickers", most of them were bullshit. In reality, this was us trying to feel better, special, doing things The Hard Way, instead of "taking shortcuts" like those lazy people... who, unlike us, actually released some playable games.

I hear echoes of this mindset in a lot of "LLMs will rot your brain" commentary these days.

--

[0] - https://news.ycombinator.com/item?id=43351486

Humans are tool users. It is very difficult to pick a point in time and say, "it was here that we crossed the Rubicon". Was it the advent of spoken word? Written word? Fire? The wheel? The horse? The axe? Or in more modern times, the automobile, calculator, or dare I say the computer and the internet?

"With the arrival of electric technology, man has extended, or set outside himself, a live model of the central nervous system itself. To the degree that this is so, it is a development that suggests a desperate suicidal autoamputation, as if the central nervous system could no longer depend on the physical organs to be protective buffers against the slings and arrows of outrageous mechanism."

― Marshall McLuhan, Understanding Media: The Extensions of Man

Are we at battle with LLMs or with humanity itself?

AI generated code? That seems a way off from being a generalized solved problem in an iterative SDLC at a modern tech company trying to get leaner, disrupt markets, and survive in a complex world. I for one am very much in support of it for engineers with the unaided experience under their belt to judge the output, but the idea that we're potentially going to train new devs at unfamiliar companies on this stuff? Yikes.

Note: I'm not saying this is a conspiracy or a cabal - it's a local phenomenon happening everywhere. Lots of people with malicious intent, and even more indifferent to the fate of others, get to make decisions hurting others, and hide behind LLCs and behind tech and technologists, all of which get blamed instead.

The value of most of us as coders will drop to zero but their skills will remain valuable for the foreseeable future. LLMs can't parrot out what's not in their training set.

LLMs are nowhere close to that level today. It spits out a bunch of mediocre code that the programmer then needs to maintain and fix bugs on. And the larger / more complicated the code base the harder it is for the LLM to work with. There's a huge leaky abstraction here going from initial vibe coding to then having to dig into the weeds, and generally fixing bugs written by another human is difficult enough, not to mention written by some random LLM.

Agreed. Most of us aren’t washing our clothes with a washboard, yet the washboard not long ago was a timesaver. Technology evolves.

Now, if AI rises up against the government and Cursor becomes outlawed, then maybe your leet coder skills will matter again.

But when a catastrophic solar storm takes out the grid, a washboard may be more useful, to give you something to occupy yourself with while the hard spaghetti of your dwindling food supply slowly absorbs tepid rainwater, and you wish that you’d actually moved to Siberia and learned to live off the land as you once considered while handling a frustrating bug in production that you could’ve solved with Claude.

No. In learning, there is no substitution to practice. Once you jump to the conclusion, you stop understanding the "why".

Throughout various stages of technological advancement, we've come with tools to help relieve us of tedious efforts, because we understood the "why", the underlying reason of what the tools were helping us solve in the first place. Those that understood the "why" could build upon those tools to further advance civilization. The others were left behind to parrot about something they did not understand.

But -and, ironically in this case- as with most things in life, it is about the journey and not the destination. And in the grand scheme of things, it doesn't really matter if we take a more efficient road, or a less efficient road, to reach a life lesson.

Enjoy your journey!

yeah you can be more effective. Just have to pay the price of becoming more stupid.

a bit hard to justify

Whereas what you're responding to is using AI to do the work for you, which weakens your own faculties the more you do it, right?

while both are a consequence of a natural or logical tendency, neither is good. Once a player optimises all the fun out of a game, they stop playing it, often without experiencing it in its entirety (which I would regard as a negative outcome for both player and game)

I am not able to extrapolate with confidence what the analogous outcome would be in the company/worker side of the equation. I would confidently say, out of just anecdotal experience, it's not trending in a good direction.

edit: typo corrections

The human brain is easily manipulated, easily hacked.

The progress comes from amplification of effot, which comes from leverage (output comes from input), not magic (output comes from nowhere) or theft (output comes from someone else).

You've arrived at the reason for compilers. You could go produce all that pesky machine code yourself or you could learn to use the compiler to its optimal potential to do the work for you.

LLMs and smart machines alike will be the same concept but potentially capable of a wider variety of tasks. Engineers that know how to wield them and check their work will see their productivity increase as the technology gets better. Engineers that don't know how to check their work or wield them will at worst get less done or produce volumes of garbage work.

It's a tradeoff. What we call progress is actually a tradeoff for what we value and prioritize more. It necessarily doesn't have to be good - just a state of the world we create.