throwuxiytayq
Your intuition that this can be balanced out is wrong. When you have a CVE in your TCP stack, your vault unlocks and you're fucked, it's done. My dude, there is never going to be AIs scanning every incoming packet for signs of a 0day, and deciding whether to, uhh, cut it up with a lightsaber and shoot it with a laser pistol. Your options are to have a stack so secure that it can't be targeted, or to disconnect all network cables.
Yes they are, and no, they don't. Debugging can be done by writing/deleting/modifying code (e.g., write a test and run it, or add log statements, or remove a line and re-run the program, etc.), or by running shell tools (either as single tool calls, which you may to be a one-line program, or with larger compound commands, or entire shell scripts - throwaway or saved to a file), or by writing code that analyzes debugger output and logs, or editing code at runtime with hot reload, or - depending on the runtime - attaching to the running program and running ad-hoc code (REPL-like), and so on. It's literally impossible to enumerate all of the options because it's all mixed in one codey-debuggy-flavored soup. Are you saying you've never seen an agent do this?
It's also fun to write your own debugging tools by the way - just imagine any debugging workflow you think is good, write the tool, give the agent some tips how to use it, and they'll do great. It's so good, you should really try it. This includes debugging tools that are operated by writing code, it totally just works.
> you cannot write code in gdb or valgrind
I'm actually pretty sure these are extensible, and I'd bet quite a lot that an LLM could write that plugin.
> The fact that you personally do both things does not mean everybody and everything does it this way.
I don't, but LLMs certainly do.
> And no, debugging and code writing are not in same datasets. More to it, debugging and architectural planning is mostly represented in fine-tune datasets (prompt-response, agentic step prediction), while code writing is mostly (like 99%) pre-training data (refined internet scraps and regenerated code).
Programming and debugging is self-evidently part of the same training loop. Agents write code, validate it and adjust trajectory. I'm not saying that they don't train agents for debugging-centric tasks specifically, but these certainly often - if not usually - involve programming as well. The same way coding tasks often involve debugging.
> Training on purely LLM-generated data is known to give horrible results.
I don't know what to tell ya, it's how the LLM sausage is made.
> `c = a + b` and `c = a + d` are similar, but the difference is very small — it's about variable rename.
I think you're vastly underestimating the difficulty of real world programming tasks. It's a pretty common mistake, I usually see this in people who are struggling to downplay LLM capabilities, and their strategy is to pretend that everything LLMs are good at was trivially accessible in the training data all along.
> No, LLM does not learn to program same way people do, it generally does not learn things the way humans do.
Obviously, it doesn't. I said "LLMs are capable of learning and extrapolating knowledge to new problems". You can throw a novel problem at an LLM and you'll get a novel solution. This is obvious to anyone who works on interesting problems. You could literally hit CTRL+T and do it right now. It's such an easily and accessibly verifiable fact. What a weird hill to die on, and yet so many corpses there.
No, they're basically the same thing, intrinsically linked and inseparable. Likely represented together in training sets, too. I'm surprised anyone would say that. Are you a programmer?
> For debugging LLM-s reproduce fine-tuning datasets made by humans actually doing debugging on some code.
Not really, they are primarily trained on LLM-generated trajectories from executable training environments.
> And yes, the problems AI found in your code were similar to the problems humans found when writing those datasets.
So they're only similar now? I suppose, in the way that I learned how to solve bugs while learning to program and now I can fix similar ones? I guess we can now agree that LLMs are capable of learning and extrapolating knowledge to new problems?
These problems looked slightly more difficult than bubble sort to me. Do you suppose these bugs existed in some other software and the agent only translated the fix between repositories? When the agent was debugging the software, were they simply following some tutorial they were trained on? Is this the 1% case? Or are the stochastic parrots regurgitating most probable tokens in approximately the right order to provide an illusion of work being done?
> In case you would say "but LLM-s learn on random source code from the internet"
Of course they don't, it's not 2022 any more.
I'm guessing you haven't had the displeasure of working for a boss who doesn't code and has no idea what he's paying you for - lucky you.
> I was questioning the absurd 10x-100x company output, which is relevant to the op.
I'm talking about programmer productivity, not company output. For pure software companies, these may be closely related, but there's many other processes involved with converting code into money. Either way, the 10x+ productivity increase is obvious to everyone I work with.
I know some people who pay for their own tokens because it lets them do their job in 30 minutes a day, and spend the rest of their time creating a side project. I know several freelancers who didn’t make software before, and now they have custom productivity tools tailored to their jobs. Others work in small teams, where the boss doesn’t need to ask for proof because they’re burning tokens themselves.
But that’s not the point here - even if this is true and it continues, IT is not the entire world economy. I don’t think we’ll see the same productivity gains propagate more widely soon. This will take years. The real world is a ship that’s immensely hard and slow to steer, which is easy to forget when your CI pipeline deploys commits to production in minutes.
But also, I read about how this tech works. Have you? If so, how are you possibly still confused about this?
It is very impressive engineering, just not very useful or ever going to result in any viable product. It’s a dead end. A waste of somebody’s engineering effort and talent for a one-time rich guy demo item. People hate on it because they get it.
Anyway, since you want a direct representation of the text - presumably, you configured your editor to render EOL sequences explicitly, to distinguish tabs from spaces, etc?
For what it’s worth, any time I see that feature, it’s all farts.
But when you squint your eyes, LLMs are just a least-effort manifestation of the wider neural net direction we’ve been chipping away at for decades. I don’t give a fuck whether LLMs are here to stay; they’re proof that scaling neural nets gets you pretty far.
The real question is: will neural nets hit a wall? As a human, I certainly hope so. But I’d bet that they won’t. I’d bet that we’re igniting the ~~atmosphere~~ universe in slow motion.
I do feel there is a limit to human-s today. I did not try, but I doubt it would work if I asked literally any programmer I know "make me a web browser that is bug free, perfectly secure, works as efficient as possible on my architecture and has best UX for me personally".
Hell, I'm willing to bet they'd fail at this task even with an unconstrained snacks and kombucha budget. Humans have a long way to go. My job is safe.
At this point I think the only reason someone would do better without an LLM is an unwillingness to use it effectively. Even if you’re going to write all code by hand and only use an agent for validation and debugging, that’s a significant boost.
Jesus Christ, I had a list of comebacks I was expecting, but “doing stuff costs money therefore we shouldn’t ask trillion dollar market cap companies to do stuff because then they have to spend a bit of their trillions of dollars” on it, somehow. Out of baseline respect for the median HN reader, I think.
> The tiny minority of HN readers that wants to run asahi linux on their mac?
Please at least try to make points worth responding to. Asahi Linux community is tiny precisely because Apple does precisely nothing to support this and similar projects. Surely you can comprehend this. I believe in you.
Although, now that I think about it, maybe NVIDIA should also be compelled to support an open-source, community-controlled driver stack. I mean, if AMD can do it, then so can they.
Laws exist to make everyone's lives better. Apple can afford it, and doing this would gift the world a breathtaking amount of value. There are no downsides.