This is obviously just a response to https://www.anthropic.com/research/redesigning-source-contro.... Probably no one in the chain of writing this even thought about your question.

Because this kind of growth is how they get to RSI.

Whereas previously, creating this amount of code by hand would simply result in RSI.

They should talk to a orthopedist, in that case.

> Technical details aside, this volume is ridiculous. How on earth do they see this kind of "growth" as an asset and not a liability?

That coding volume is a liability only for human engineers, with their limited memory, short attention span, tiredness, and quirky coding preferences. LLMs don't have any of those problems.

[delayed]

who cares what the llm is doing? most of the time when there is disagreement, the llm is right and you are wrong anyways.

How would you know?

How do I know my calculator is outputting the correct answer to a 10 by 10 digit multiplication problem?

Because someone understood and verified it.

And that some one can be an LLM. Frankly, these discussions are tiring because there's a false notion that humans have to be in the loop 100% of the time. This is untrue. LLMs are accurate enough that they can be left to do things on their own and human intervention is needed only for out of distribution instances.

Obviously there will be a maximum hit and then there will be no one to ask to solve your problem and youre gonna be up a creek

> Does your manager at your job review every line of code you wrote or trust you based on your competence and reviews only a compressed portion?

The manager always has a very good stick: “Do a good job or you will be fired”. And me wanting food is well aligned to not produce buggy code. Or make my job harder in the future by writing unmaintainable code.

> Frankly, these discussions are tiring because there's a false notion that humans have to be in the loop 100% of the time

The code is written to process data. Someone needs to be aware of how it’s processed, so that when malformed data appears, we can collectively find where it happens. You can always play around with your own data (if regulation allows) but you need to respect your customers’ data. Not looking at the code jis just negligence.

> The manager always has a very good stick: “Do a good job or you will be fired”. And me wanting food is well aligned to not produce buggy code. Or make my job harder in the future by writing unmaintainable code.

You are welcome to not use the model if it makes mistakes. Its weird that you are cooking up some edge case scenario to make your argument strong - everyone is incentivised to make things work well. Anthropic wants their models to perform well because that's why you will buy it.

> Someone needs to be aware of how it’s processed, so that when malformed data appears

Eh no. You need to know about only a compressed part of it, not 100%. Do you really claim that you need to know 100% of the code? This is not true today and certainly won't be true in the future. Hardly any code will be looked at by humans in the same way my manager doesn't look at my code.

> Anthropic wants their models to perform well because that's why you will buy it.

What Anthropic wants and what happens in reality are two different things. But with code I write, I do control the process.

> Eh no. You need to know about only a compressed part of it, not 100%. Do you really claim that you need to know 100% of the code?

I know the code I write, and I know how it interfaces with external modules. When those interfaces don’t function as described, I look at the code if I have access to it. When I don’t, the party in control has always good incentives to produce good documentation. That knowledge is usually good for some time (years even)

> my manager doesn't look at my code

Because that’s not his responsibility, just like he’s not responsible for bugs and crashes. He’s responsible for the project and the team and you can be pretty sure that he’s looking at all at the information about those. Imagine if his own manager ask him about some details and he answers: “I don’t know, Claude is doing it”.

Didn't the Postgres team just have to stop a release because the agents found a slew of bugs in human code from domain experts that "understood and verified it"?

Perhaps I'm being overly generous to the parent's point, but it stands to reason that we make a lot of choices of where and how much trust we give away. This is just another one of those points.

I would say that a calculator and a database like postgres differ quite significantly, though. How many PRs are merged per month on your average calculator? How many lines of code is in that firmware?

Trusting that a calculator works is like trusting that there will be another day tomorrow, while trusting the code of an LLM is like trusting that the current weather will hold for the next hour.

Also, LLM finding bugs in human written code doesn't mean it wouldn't find much more in LLM generated code, and spend much more money doing so. That is, of course, after ruling out the false positives (of which there are a few, judging by the PRs I've received on my open source projects)

It's an LLM. It is by definition smarter than you, and my shareholders know it.

Not necessarily. High LoC software is a potential bad design smell (without a counterfactual for the same level of functionality it might not be easy to tell though). Badly designed code entering an agent context is always a liability, because it contaminates decisions down the road.

Every bit of code is more context getting soaked up in future agent runs. More context used = worse output. More tests need to be written, then fixed, using up more compute, time and context without adding value compared to a more thoughtful, efficient implementation.

> using up more compute, time and context

LLM compute is getting cheaper, so one can re-run the coding agent until it produces the desired result.

> without adding value compared to a more thoughtful, efficient implementation

Thoughtful implementations require thoughtful developers, and thoughtful developers cost more money that dumb GPUs.

It's not just the GPUs that I was referring to, but also the time spent in test runners locally and in CI. There's also the possible runtime cost of extraneous abstractions, but that's beside the point I was getting at.

Thoughtful developers do take money, but so too does shitty code that reaches a point LLMs struggle to maintain. Having both- with the thoughtful developers in charge- is better than just throwing more LLMs at it and hoping for the best.

You can't make these argument against the position that all these problems will be solved. Any problem you point to, someone would argue that an LLM can solve. Expensive compute? costs could get 1000x cheaper. Longer context gets worse output? Newer models would solve that. Ever increasing massive codebase? We'll get a 1B or a 1T context window. Time spent in test runners? parallelize, or selectively select tests, or even better; ask the LLM to optimize the tests or develop a 100x faster chips. Runtime cost of extraneous abstractions? Ask the LLM to keep functionality and focus on performance.

If you seriously believe we have "solved coding" or achieved AGI in coding, what is not possible? Any argument you can come up with will be easily dismissed with "we'll just ask the AGI to solve that problem".

The difference is the first post I replied to used present tense. None of the magic wand thinking applies to the reality of today.

I'm not sure why this is downvoted. This is the correct answer. Code is mostly written by and for agents with the option for humans to read it from time to time.

This kind of fallacy keeps repeating time and time again. Another version of this fallacy goes like this: "agents can create a lot of code but humans have to review each PR anyway so reviewing is a massive bottleneck". This doesn't make sense because reviewing is also done by agents and humans don't have to read every line.

I think you guys forget that ultimately the goal of products is to have people use them.

Who is this code helping?

Instead of keeping it simple, they decided to go stupid.

It's an asset because it's a demonstration that their technology works. It would be stupid for them not to tokenmaxx even if that was more efficient. Kinda like you don't want an Apple salesman to be using an Android device.

This thing aside... IDK, I used to think that there's no way to manage such a mess but apparently there is. Or rather, it starts to seem plausible that models will evolve faster than this mess, and once it becomes unmanageable, you can simply ask AI "make it better, no bugs please" and it'll mostly work.

It's becoming harder and harder to justify my own existence as a software engineer. It's surreal that that I dedicated years of study to prepare myself for a job that's being automated away.

Of course the demand for human developers won't drop to 0 but it'll be limited to a handful of industries where even small mistakes are costly, like aerospace. But then let's be real - what percentage of currently employed devs are competent enough to write such code?

I have full faith in LLMs to produce code that does something (whether or not it's desired or expected). I have little faith that the engineers (or MBAs) have any clue what they're doing. It'll be interesting to see what parts still exist in ten years and what parts they'll have to light on fire to contain bad assumptions.

Not really. In fact it's fairly mild given OpenAI's size. For my personal work I've stopped even counting code changes in locs/commits and started looking at disk size changes.