News for hackers. Stuff that matters.
Re:Two things LLM coding agents are still bad at by hansmayer on Thursday October 09, 2025 @03:46AM (#45524745)
Yeah so, the reason people use various tools and machines in the first place is to simplify the work or everydays tasks by : 1) Making the tasks execute faster 2) Getting more reliable outputs then doing this by yourself 3) Making it repeatable . The LLMs obviously dont check any of these boxes so why don´t we stop pretending that we as users are stupid and don´t know how to use them and start taking them for what they are - cute little mirages, perhaps applicable as toys of some sort, but not something we should use for serious engineering work really?
Re:Two things LLM coding agents are still bad at by ivape on Thursday October 09, 2025 @03:52AM (#45524792)
Just stop. You need to take some time and run some Google searches before 2022 on HN and just read how blind sided this community has been by AI. Take the L, like everyone else.

Quite frankly, discussion about LLMs being hype is not useful for an innovation focused community, especially when it’s utterly wrong. The AI-is-hype people are so wrong, oh so wrong. This will be fun to revisit in just a year.

Re:Two things LLM coding agents are still bad at by exe34 on Thursday October 09, 2025 @03:59AM (#45524840)
I take it you have bought stocks? What do you recommend?
Re:Two things LLM coding agents are still bad at by jsjshxnsbs on Thursday October 09, 2025 @04:18AM (#45524956)
I'll take the L when llms can actually do my job to the level I expect. Llms can do some of my work but they are tiring they make mistakes and they absolutely get confused by a sufficiently complex and large codebase.

Quite frankly, not being able to discuss the pros and the cons of a technology with other engineers absolutely hinders innovation. A lot of discoveries come out of mistakes.

Stop being so small minded.

Re:Two things LLM coding agents are still bad at by laterium on Thursday October 09, 2025 @02:19PM (#45531215)
Why is the bar for it to do your job or completely replace you? It's a tool. If it makes you 5% better at your job, then great. There's a recent study showing it has 15-20% productivity benefits: not completely useless, not 10x. I hope we can have nuance in the conversation.
Re:Two things LLM coding agents are still bad at by hansmayer on Friday October 10, 2025 @08:07AM (#45538000)
...and then there was also a recent MIT study showing it was making everyone less productive. The bar is there because this is how all the AI grifters have been selling this technology - no less than end of work itself. Why should we not hold them accountable for over-promising and under-delivering? Or is that reserved just for the serfs?
Re:Two things LLM coding agents are still bad at by hansmayer on Thursday October 09, 2025 @04:45AM (#45525126)
Stop what mate? My words are not the words of someone who ocassionally dabbles in the free ChatGPT layer - I've been paying premium tier AI tools for my entire company for a long time now. Recently we had to scale back their usage to just consulting mode, i.e. because the agent mode has gone from somewhat-useful to complete waste of time. We are now back to using them as replacement for the now entshittified search. But as you can see by my early adopting of these crap-tools, I am open-minded. I'd love to see what great new application you have built using them. But if you don't have anything to show, I'll also take some arguments, you know, like the stuff I provided in my first comment.
Re:Two things LLM coding agents are still bad at by IanCal on Thursday October 09, 2025 @08:39AM (#45526801)
They easily check a bunch of those boxes.

> why don´t we stop pretending that we as users are stupid and don´t know how to use them

This is in response to someone who saw a bunch of URLs coming out of it and was surprised at a bunch of them being wrong. That's using the tool wrong. It's like being surprised that the top results in google/app store/play store aren't necessarily the best match for your query but actually adverts!

Re:Two things LLM coding agents are still bad at by hansmayer on Thursday October 09, 2025 @09:08AM (#45527183)
The URLs being wrong in that specific case is one where they were using the "wrong tool". I can name you at least a dozen other cases from own experience, where too, they appear to be the wrong tool, for example for working with Terraform or for not exposing secrets by hardcoding them in the frontend. Et cetera. Many other people will have contributed thousands if not more similar but different cases. So what good are these tools then for really? Are we all really that stupid? Many of us mastered the hard problem of navigating various abstraction layers of computer over the years, only to be told, we now effing dont know how to write a few sentences in English? Come on. I'd be happy to use them in whatever specific domain they supposedly excel at. But no-one seems to be able to identify one for sure. The problem is, the folks pushing or better said, shoving down these bullshit generators down our throats are trying to sell us the promise of an "everything oracle". What did old man Altman tell us about ChatGPT 5? PhD level tool for code generation or some similar nonsense? But it turns out it only gets one metric right each time - generating a lot of text. So, essentially, great for bullshit jobs (i count some of the IT jobs as such too), but not much more.
Re:Two things LLM coding agents are still bad at by IanCal on Thursday October 09, 2025 @09:41AM (#45527569)
> Many of us mastered the hard problem of navigating various abstraction layers of computer over the years, only to be told, we now effing dont know how to write a few sentences in English? Come on.

If you're trying to one shot stuff with a few sentences then yes you might be using these things wrong. I've seen people with PhDs fail to use google successfully to find things, were they idiots? If you're using them wrong you're using them wrong - I don't care how smart you are in other areas. If you can't hand off work knowing someones capabilities then that's a thing you can't do - and that's ok. I've known unbelievably good engineers who couldn't form a solid plan to solve a business problem or collaboratively work to get something done to save their life. Those are different skills. But gpt5-codex and sonnet 4 / 4.5 can solidly write code, gpt-5-pro with web search can really dig into things, and if you can manage what they can do you can hand off work to them. If you've only ever worked with juniors with a feeling of "they slow everything down but maybe someday they'll be as useful as me" then you're less likely to succeed at this.

Let's do a quick overview of recent chats for me:

* Identifying and validating a race condition in some code

* Generating several approaches to a streaming issue, providing cost analyses of external services and complexity of 3 different approaches about how much they'd change the code

* Identifying an async bug two good engineers couldn't find in a codebase they knew well

* Finding performance issues that had gone unnoticed

* Digging through synapse documentation and github issues to find a specific performance related issue

* Finding the right MSC for a feature I wanted to use but didn't know existed - and then finding the github issue that explained how it was only half implemented and how to enable the experimental other part I needed

* Building a bunch of UI stuff for a short term contract I needed, saving me a bunch of hours and the client money

* Going through funding opportunities and matching them against a charity I want to help in my local area

* Building a search integration for my local library to handle my kids reading challenge

* Solving a series of VPN issues I didn't understand

* Writing a lot of astro related python for an art project to cover the loss of some NASA images I used to have access to.

> the folks pushing or better said

If you don't want to trust them, don't. Also don't believe the anti-hype merchants who want to smugly say these tools can't do a god damn thing. They're trying to get attention as well.

Re:Two things LLM coding agents are still bad at by hansmayer on Thursday October 09, 2025 @10:03AM (#45527877)
Again mate, stop making arrogant assumptions and read some of my previous comments. I and my team are early adopters, since about 2 years. I am even paying for premium-level service. Trust me, it sucks and under-delivers. But good for you and others who claim they are productive with it - I am sure we will see those 10x apps rolling in soon, right? It's only been like 4 years since the revolutionary magic machine was announced.
Re:Two things LLM coding agents are still bad at by IanCal on Thursday October 09, 2025 @10:28AM (#45528191)
I read your comments. Did you read mine? You can pass them into chatgpt or claude or whatever premium services you pay for to summarise them for you if you want.

> Trust me, it sucks

Ok. I'm convinced.

> and under-delivers.

Compared to what promise?

> I am sure we will see those 10x apps rolling in soon, right?

Did I argue that? If you want to look at some massive improvements, I was able to put up UIs to share results & explore them with a client within minutes rather than it taking me a few hours (which from experience it would have done).

> It's only been like 4 years since the revolutionary magic machine was announced.

It's been less than 3 since chatgpt launched, which if you'd been in the AI sphere as long as I had (my god it's 20 years now) absolutely was revolutionary. Over the last 4 years we've seen gpt3 solve a bunch of NLP problems immediately as long as you didn't care about cost to gpt-5-pro with web search and codex/sonnet being able to explore a moderately sized codebase and make real and actual changes (running tests and following up with changes). Given how long I spent stopping a robot hitting the table because it shifted a bit and its background segmentation messed up, or fiddling with classifiers for text, the idea I can get a summary from input without training is already impressive and then to be able to say "make it less wanky" and have it remove the corp speak is a huge shift in the field.

If your measure of success is "the CEOs of the biggest tech orgs say it'll do this soon and I found a problem" then you'll be permanently disappointed. It'd be like me sitting here saying mobile phones are useless because I was told how revolutionary the new chip in an iphone was in a keynote.

Since you don't seem to want to read most of this, most isn't for you. The last bit is, and it's just one question:

Why are you paying for something that solves literally no problems for you?

Re:Two things LLM coding agents are still bad at by hitarpetar on Thursday October 09, 2025 @11:37AM (#45529198)
it's amazing that you picked another dark pattern as your comparison
Re:Two things LLM coding agents are still bad at by mbesto on Thursday October 09, 2025 @11:27AM (#45529054)
> This is in response to someone who saw a bunch of URLs coming out of it and was surprised at a bunch of them being wrong. That's using the tool wrong. It's like being surprised that the top results in google/app store/play store aren't necessarily the best match for your query but actually adverts!

The CEO of Anthropic said I can fire all of my developers soon. How could one possibly be using the tool wrong? /s

Re:Two things LLM coding agents are still bad at by IanCal on Thursday October 09, 2025 @11:51AM (#45529390)
If you base all your tech workings on the promises of CEOs you'll fail badly, you should not be surprised by this.
Re:Two things LLM coding agents are still bad at by mbesto on Friday October 10, 2025 @09:01AM (#45538519)
Thanks for the advice...