News for hackers. Stuff that matters.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by gyomu on Monday May 26, 2025 @06:27AM (#44095941)
There are two kinds of engineers.

Those who can’t stop raving about how much of a superpower LLMs are for coding, how it’s made them 100x more productive, and is unlocking things they could’ve never done before.

And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best.

The only thing I don’t understand is why people from the former group aren’t all utterly dominating the market and obliterating their competitors with their revolutionary products and blazing fast iteration speed.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by Philip-J-Fry on Monday May 26, 2025 @07:10AM (#44096206)
I find LLMs 100x more productive for greenfield work.

If I want to create a React app with X amount of pages, some Redux stores, Auth, etc. then it can smash that out in minutes. I can say "now add X" and it'll do it. Generally with good results.

But when it comes to maintaining existing systems, or adding more complicated features, or needing to know business domain details, a LLM is usually not that great for me. They're still great as a code suggestion tool, finishing lines and functions. But as far as delivering whole features, they're pretty useless once you get past the easy stuff. And you'll spend as much time directing the LLM to do this kind of this as you would just writing it yourself.

What I tend to do is write stubbed out code in the design I like, then I'll get an LLM to just fill in the gaps.

These people who say LLMs make them 100x more productive probably are only working on greenfield stuff and haven't got to the hard bit yet.

Like everyone says, the first 90% is the easy bit. The last 10% is where you'll spend most of your time, and I don't see LLMs doing the hard bit that well currently.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by virgilp on Monday May 26, 2025 @08:15AM (#44096707)
I've recently been able to use LLM on a large-ish internal project to find a bug. The prompt took the form of "here's the symptoms I observe, and some hypothesis, tell me where the code that handles this case is written" (it was a brand new repo that I hadn't looked at before - code written by a different team, that were claiming some weird race condition/ were not really willing to look into the bug). Basically I was asking the LLM to tell me where to look, and it actually found the issue itself.

Not 100x more productive, that's an exaggeration... not even 10x. But it helps. It is an extremely competent rubber duck [1].

[1] https://en.wikipedia.org/wiki/Rubber_duck_debugging

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by tormeh on Monday May 26, 2025 @08:56AM (#44097037)
I too did this (although on a small project), and I was incredibly impressed. My problem with it is that I first did it myself, and it was fairly quick and easy. The hard part was figuring out that there was a bug, and how exactly the bug behaved. The LLM helped with the easy part, but I don't know how to even explain the difficult part to it. There was no way to know which repo the problem is in, or that it wasn't a user error.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by joshstrange on Monday May 26, 2025 @07:21AM (#44096273)
I couldn’t agree more and I’ve said the same thing many times.

I have yet to see an LLM-generated app not collapse under it’s own weight after enough iterations/prompts. It gets stuck in loops (removing and adding the same code/concept), it gets hung up on simple errors, etc.

For greenfield it’s amazing, no doubt, but unless you are watching it very closely and approving/reviewing the code along the way it will go off the rails. At a certain point it’s easier to add the new feature or make the modification yourself. Even if the LLM could do it, it would burn tons of money and time.

I expect things to get better, this will not always be the state of things, but for now “vibe coding” (specifically not reviewing/writing code yourself) is not sustainable.

Most people doing it have a github profile that is a mile wide and a meter deep.

LLM’s are amazing and useful, but “vibe coding” with them is not sustainable currently.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by RajT88 on Monday May 26, 2025 @10:02AM (#44097509)
Coding LLM's: Ruby off rails?
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by bredren on Monday May 26, 2025 @04:40PM (#44101460)
The way to get past the removing and adding is to have the prompt include what has already been done and what is left to do.

Then specify the need to conclude the work by a deadline.

These kinds of things cause the LLM to “finish” tasks and try to move on or say it is done when it is.

This won’t let you skip the curation of output along the way, but at least some of the stumbling around can be handled with prompting.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by noosphr on Monday May 26, 2025 @10:32PM (#44103459)
>I expect things to get better, this will not always be the state of things, but for now “vibe coding” (specifically not reviewing/writing code yourself) is not sustainable.

It will not.

And I say this as someone whose been building internal LLM tools since 2021.

The issue is their context window. If you increase the context window so they can see more code costs skyrocket as n^2 the size of the code base. If you don't then you have all the issues people have in this thread.

The reason why I have a job right now is that you can get around this by building tooling for intelligent search that limits the overfill of each context window. This is neither easy, fast, or cheap when done at scale. Worse the problems that you have when doing this are at best very weakly related to the problems the major AI labs are focusing on currently - I've interviewed at two of the top five AI labs and none of the people I talked to cared or really understood what a _real_ agentic system that solves coding should look like.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by quartzeee on Tuesday May 27, 2025 @08:31AM (#44106301)
> building tooling for intelligent search that limits the overfill of each context window

I'm interested to know what you mean by this, in our system we've been trying to compress the context but this is the first I've seen about filtering it down.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by noosphr on Tuesday May 27, 2025 @10:32AM (#44107379)
For general text you run some type of vector search against the full-text corpus to see what relevant hits there are and where. Then you feed the first round of results into a ranking/filtering system which does pair wise comparison between each chunk that you've had a good score from the vector search. Contract/expand until you've reach the limit of the context window for your model and run against the original query.

For source code, you are even luckier since there are a lot of deterministic tools which provide solid grounding, e.g., etags, and the languages themselves enforce a hierarchical tree-like structure on the source code, viz. block statements. The above means that ranking and chunking strategies are solved already - which is a huge pain for general text.

The vector search is then just an enrichment layer on top which brings in documentation and other soft grounding text that keeps the LLM from going berserk.

Of course, none of the commercial offerings come even close to letting you do this well. Even the dumb version of search needs to be a self-recursive agent which comes with a good set of vector embeddings and the ability to decide if it's searched enough before it starts answering your questions.

If you're interested drop a line on my profile email.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by FLT8 on Tuesday May 27, 2025 @06:52PM (#44111374)
I can't help but wonder whether the solution here is something like building a multi-resolution understanding of the codebase. All the way from an architectural perspective including business context, down to code structure & layout, all the way down to what's happening in specific files and functions.

As a human, I don't need to remember the content of every file I work on to be effective, but I do need to understand how to navigate my way around, and enough of how the codebase hangs together to be able to make good decisions about where new code belongs, when and how to refactor etc.. I'm pretty sure I don't have the memory or reading comprehension to match a computer, but I do have the ability to form context maps at different scales and switch 'resolution' depending on what I'm hoping to achieve.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by cwsx on Tuesday May 27, 2025 @03:55AM (#44104827)
> For greenfield it’s amazing

I'll preface this comment with: I am a recent startup owner (so only dev, which is important) and my entire codebase has been generated via Sonnet (mostly 3.7, now using 4.0). If you actually looked at the work I'm (personally) producing, I guess I'm more of a product-owner/project-manager as I'm really just overseeing the development.

> I have yet to see an LLM-generated app not collapse under it’s own weight after enough iterations/prompts.

There's a few crucial steps to make an LLM-generated app maintainable (by the LLM):

- _have a very, very strong SWE background_; ideally as a "strong" Lead Dev, _this is critical_

- your entire workflow NEEDS to be centered around LLM-development (or even model-specific):

  - use MCPs wherever possible and make sure they're specifically configured for your project

  - don't write "human" documentation; use rule + reusable prompt files

  - you MUST do this in a *very* granular but specialized way; keep rules/prompts very small (like you would when creating tickets)

  - make sure rules are conditionally applied (using globs); do not auto include anything except your "system rules"

  - use the LLM to generate said prompts and rules; this forces consistency across prompts, very important

  - follow a typical agile workflow (creating epics, tickets, backlogs etc)

  - TESTS TESTS AND MORE TESTS; add automated tools (like linters) EVERYWHERE you can

  - keep your code VERY modular so the LLM can keep a focused context, rules should provide all key context (like the broader architecture); the goal is for your LLM to only need to read or interact with files related to the strict 'current task' scope

  - iterating on code is almost always more difficult than writing it from scratch: provided your code is well architected, no single rewrite should be larger than a regular ticket (if the ticket is too large then it needs to be split up)
This is off the top of my head so it's pretty broad/messy but I can expand on my points.

LLM-coding requires a complete overhaul of your workflow so it is tailored specifically to an LLM, not a human, but this is also a massive learning curve (that take's a lot of time to figure out and optimize). Would I bother doing this if I were still working on a team? Probably not, I don't think it would've saved me much time in a "regular" codebase. As a single developer at a startup? This is the only way I've been able to get "other startup-y" work done while also progressing the codebase - the value of being able to do multiple things at a time, let the LLM and intermittently review the output while you get to work on other things.

The biggest tip I can give: LLMs struggle at "coding like a human" and are much better at "bad-practice" workflows (e.g. throwing away large parts of code in favour of a total rewrite) - let the LLM lead the development process, with the rules/prompts as guardrails, and try stay out of it's way while it works (instead of saying "hey X thing didn't work, go fix that now") - hold its hand but let it experiment before jumping in.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by tmaly on Tuesday May 27, 2025 @03:05PM (#44109797)
Do you have an example of a rule file? Or the MCPs you use?
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by cwsx on Tuesday May 27, 2025 @04:07PM (#44110249)
MCPs:

  - `server-sequential-thinking` (MVP)
  - `memory` (2nd MVP, needs custom rules for config)
  - `context7`
  - `filesystem`
  - `fetch`
  - `postgres`
  - `git`
  - `time`

Example rules file for ticketing system:

```

# Ticket Management Guidelines

This document outlines the standardized approach to ticket management in the <redacted> project. All team members should follow these guidelines when creating, updating, or completing tickets.

## Ticket Organization

Tickets are organized by status and area in the following structure:

TICKETS/ COMPLETED/ - Finished tickets BACKEND/ - Backend-related tickets FRONTEND/ - Frontend-related tickets IN_PROGRESS/ - Tickets currently being worked on BACKEND/ FRONTEND/ BACKLOG/ - Tickets planned but not yet started BACKEND/ FRONTEND/

## Ticket Status Indicators

All tickets must use consistent status indicators:

- *BACKLOG* - Planned but not yet started - *IN_PROGRESS* - Currently being implemented - *COMPLETED* - Implementation is finished - *ABANDONED* - Work was stopped and will not continue

## Required Ticket Files

Each ticket directory must contain these files:

1. *Main Ticket File* (TICKET_.md): - Problem statement and background - Detailed analysis - Implementation plan - Acceptance criteria

2. *Implementation Plan* (IMPLEMENTATION_PLAN.md): - Detailed breakdown of tasks - Timeline estimates - Success metrics

3. *Implementation Progress* (IMPLEMENTATION_PROGRESS.md): - Status updates - Issues encountered - Decisions made

4. *Design Documentation* (DESIGN_RECOMMENDATIONS.md), when relevant: - Architecture recommendations - Code patterns and examples - Error handling strategies

5. *API Documentation* (API_DOCUMENTATION.md), when applicable: - Interface definitions - Usage examples - Configuration options

## Ticket Workflow Rules

### Creating Tickets

1. Create tickets in the appropriate BACKLOG directory 2. Use standard templates from .templates/ticket_template.md 3. Set status to *Status: BACKLOG* 4. Update the TICKET_INDEX.md file

### Updating Tickets

1. Move tickets to the appropriate status directory when status changes 2. Update the status indicator in the main ticket file 3. Update the "Last Updated" date when making significant changes 4. Document progress in IMPLEMENTATION_PROGRESS.md 5. Check off completed tasks in IMPLEMENTATION_PLAN.md

### Completing Tickets

1. Ensure all acceptance criteria are met 2. Move the ticket to the COMPLETED directory 3. Set status to *Status: COMPLETED* 4. Update the TICKET_INDEX.md file 5. Create a completion summary in the main ticket file

### Abandoning Tickets

1. Document reasons for abandonment 2. Move to COMPLETED/ABANDONED directory 3. Set status to *Status: ABANDONED* 4. Update the TICKET_INDEX.md file

## Ticket Linking

When referencing other tickets, use relative links with appropriate paths:

markdown @TICKET_NAME

Ensure all links are updated when tickets change status.

## Ticket Cleanup and Streamlining

### When to Streamline Tickets

Tickets should be streamlined and cleaned up at major transition points to maintain focus on remaining work:

1. *Major Phase Transitions* - When moving between phases (e.g., from implementation to testing) 2. *Milestone Achievements* - After completing significant portions of work (e.g., 80%+ complete) 3. *Infrastructure Readiness* - When moving from setup/building to operational phases 4. *Team Handoffs* - When different team members will be taking over the work

### What to Streamline

*Replace Historical Implementation Details With:* - Brief completed tasks checklist ( high-level achievements) - Current status summary - Forward-focused remaining work

*Remove or Simplify:* - Detailed session-by-session progress logs - Extensive implementation decision histories - Verbose research findings documentation - Historical status updates and coordination notes

### Why Streamline Tickets

1. *Git History Preservation* - All detailed progress, decisions, and implementation details are preserved in git commits 2. *Clarity for Future Work* - Makes it easier to quickly understand "what needs to be done next" 3. *Team Efficiency* - Anyone picking up the work can immediately see current state and next steps 4. *Maintainability* - Shorter, focused tickets are easier to read, understand, and keep updated

### How to Streamline

1. *Archive Detailed Progress* - Historical implementation details are preserved in git history 2. *Create Completion Summary* - Replace detailed progress with a brief "What's Complete" checklist 3. *Focus on Remaining Work* - Make current and future phases the primary content 4. *Update Status Sections* - Keep status concise and action-oriented 5. *Preserve Essential Context* - Keep architectural decisions, constraints, and key requirements

*Goal*: Transform tickets from "implementation logs" into "actionable work plans" while preserving essential context.

## Maintenance Requirements

1. Keep the TICKET_INDEX.md file up to date 2. Update "Last Updated" dates when making significant changes 3. Ensure all ticket files follow the standardized format 4. Include links between related tickets in both directions

## Complete Documentation

For detailed instructions on working with tickets, refer to:

- @Ticket Workflow Guide - @Ticket Index - @Tickets README

```

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by gedy on Monday May 26, 2025 @07:23AM (#44096298)
> If I want to create a React app with X amount of pages, some Redux stores, Auth, etc. then it can smash that out in minutes. I can say "now add X" and it'll do it. Generally with good results.

Not discounting your experience, but a lot of these examples are about frameworks that never had good bootstrapping, such as Rails does/did. LLMs are really good at boilerplate, but maybe this points to these such stacks needing too much fiddling to get going, vs 10x coder AI.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by Jensson on Monday May 26, 2025 @08:01AM (#44096600)
You can get similar results by downloading a boilerplate template project for that though.

LLM makes that a bit easier and faster to do, but is also a bit more error prone than a template project.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by cowsandmilk on Monday May 26, 2025 @08:14AM (#44096694)
I’m not sure that LLM makes it easier to do. Pain points I’ve seen:

1. You have to remember all the technologies you need included, my company template already has them.

2. LLM doesn’t have a standardized directory structure, so you end up with different projects having different structures and file naming conventions. This makes later refactoring or upgrades across multiple projects less automatable (sometimes this can be solved by having an LLM do those, but they often are unsuccessful in some projects still)

3. LLMs have a knowledge cutoff. If your company has already moved to a version after that knowledge cutoff, you need to upgrade the LLM generated code.

I very much prefer having a company template to asking an LLM to generate the initial project.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by gedy on Monday May 26, 2025 @08:20AM (#44096733)
I agree, initial comment was basically that, and seeing a lot of folks (especially those with debatable technical skills) being very impressed with LLMs for boilerplate generation, e.g. "I build a WHOLE APP", etc.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by skydhash on Monday May 26, 2025 @03:48PM (#44100967)
Maybe they haven't use the wizard in IDEs like Intellij and Visual Studio. You can boostrap things so quickly that you don't even think about it, just like creating a new file in the editor.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by wejick on Monday May 26, 2025 @08:52AM (#44096994)
Not really my case. Found that Codebase with good code benefit more with LLM, but it's not the prerequisite.

I just rewrote 300ish advanced PostgreSQL queries to mysql queries. The process is not magical, but it will take me 1 week rather than 3 days. Now I'm on testing phase, seems promising.

The point is if we can find a work to work along with the agent, can be very productive.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by daveguy on Monday May 26, 2025 @11:20AM (#44098312)
> I just rewrote 300ish advanced PostgreSQL queries to mysql queries.

Translation from one set of tokens to another is exactly the primary use case of LLMs. This is exactly what it should be good at. Developing new queries, much less so. Translation from one set of database queries to another was already very well defined and well covered before LLMs came about.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by skydhash on Monday May 26, 2025 @03:52PM (#44101030)
Also, both are formal grammar so if you really wanted to create a 1:1 translator, it's possible to do so (see virtual machines). But it's not as useful per se as no one really switch databases on a whim. If you really want to do so, you want to do it correctly.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by theappsecguy on Tuesday May 27, 2025 @09:52AM (#44107028)
I mean, using a good ORM with DB adapter options could achieve this in minutes. Sure, LLM has utility here for raw queries, hardly "replace SWEs" type of utility though
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by DonHopkins on Tuesday May 27, 2025 @11:22AM (#44107840)
A good reason to have "300ish advanced PostgreSQL queries" is because an "ORM with DB adapter" is inadequate for your requirements.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by bonki on Tuesday May 27, 2025 @11:13PM (#44112459)
Why did you not use an SQL transpiler?
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by zmgsabst on Monday May 26, 2025 @08:55AM (#44097026)
I don’t find the same, eg, greenfield AI projects.

It can do pieces in isolation, but requires significant handholding on refactors to get things correct (ie, it’s initial version has subtle bugs) — and sometimes requires me to read the docs to find the right function because it hallucinates things from other frameworks will work in its code.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by sph on Monday May 26, 2025 @10:18AM (#44097662)
> I find LLMs 100x more productive for greenfield work.

Greenfield != boilerplate and basic CRUD app.

I'm a consultant writing greenfield apps solo, and 90% of my time is spent away from my editor thinking, planning, designing, meeting with stakeholders. I see no benefit in using a low-IQ autocomplete tool to automate a small part of the remaining 10% of the job, the easiest and most enjoyable part in fact.

Personally I find most of coding I do is unsuitable for LLMs anyway, because I don't need them to regurgitate standard logic when libraries are available, so most of that 10% is writing business logic tailored for the program/client.

Call me elitist (I don't care) but LLMs are mostly useful to two kinds of people: inexperienced developers, and those that think that hard problems are solved with more code. After almost two decades writing software, I find I need fewer and fewer code to ship a new project, most of my worth is thinking hard away from a keyboard. I really don't see the use of a machine that egregiously and happily writes a ton of code. Less is more, and I appreciate programming-as-an-art rather than being a code monkey paid by the line of code I commit.

Disclaimer: I am anti-LLM by choice so my bias is opposite than most of HN.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by bigtex on Monday May 26, 2025 @12:05PM (#44098700)
You shared my sentiments exactly. What none of these "all junior developers will be out of a job by 2026" proclamations never deal with is the non coding stuff. Sure it can generate a boiler plate app in 1.5 seconds, but can in communicate with a stakeholder about the requirements and ask the right questions to determine the scope and importance of features.? I just imagine my Sales President spending more 90 seconds trying to write the proper prompt before he gives up and calls and talks to a human. There is just no way a C-suite executive is sitting in front of a computer typing in prompts.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by ungreased0675 on Monday May 26, 2025 @10:33PM (#44103466)
I agree with the larger point you’re making, but an LLM absolutely can ask the right questions to determine scope and features for a basic application. It can even turn those into decent user stories. It’s the edge cases and little details that will be messed up.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by s_ting765 on Monday May 26, 2025 @10:52AM (#44097997)
I completely agree with you that the coders who are "smitten" by LLMs are just inexperienced. I personally find that LLMs get subtle improvements in capability over time and it's usually worthwhile to check in on the progress from time to time. Even if it's just for fun.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by l33tman on Monday May 26, 2025 @11:14AM (#44098254)
I can see why you would chime in to say that in your experience you don't get any value out of it, but to chime in to say that the millions of people who do are "inexperienced" is pretty offensive. In the hands of skilled developers these tools are a complete gamechanger.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by bluefirebrand on Monday May 26, 2025 @11:47AM (#44098529)
> In the hands of skilled developers these tools are a complete gamechanger.

This is where both sides are basically just accusing the other of not getting it

The AI coders are saying "These tools are a gamechanger in the hands of skilled developers" implying if you aren't getting gamechanging results you aren't skilled

The non-AI coders are basically saying the same thing back to them. "You only think this is gamechanging because you aren't skilled enough to realize how bad they are"

Personally, I've tried to use LLMs for coding quite a bit and found them really lacking

If people are finding a lot of success with them, either I'm using them wrong and other people have figured out a better way, or their standards are way, way lower than mine, or maybe they wind up spending just as long fixing the broken code as it would take me to write it

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by l33tman on Friday May 30, 2025 @05:14AM (#44134365)
Sorry for the late reply here, but let me develop a bit more what I meant. I have a long and vast programming experience, but am also an entrepreneur with many projects that I switch between. I could have focused full-time on, say, React Native, and then I could churn out my monkey code all day long. But I don't spend full-time on RN and then suddenly 12 months passed where I didn't write any at all so my specific knowledge of that domain is always a bit behind.

But something like o4-mini-high is a domain expert in all versions of React, Redux, RN etc. and knows every internal SDK change over the last 10 years (or goes out and reads the changelogs and code itself). Countless times I've had it port old code to new and it figures it out 100%. It formulates good modern canonical ways to solve stuff. It knows all the stupid tricks you have to do to get RN stuff run well on Android and iOS that I would never be able to keep in my head unless I work full-time on that. And it does the eye-watering boring styling code that nobody likes, you can even just upload a screenshot of another app or a sketch on paper and it will correctly output code for the style in a matter of seconds.

The end result is that I can, without investing a full-time of keeping myself current, do a professional RN dual Android/iOS app development cycle because I have the general skill to understand what to ask it and how to merge its output properly. This leaves me time to do other stuff and generally be more productive.

My guess is that many who gave up on the AI coding stuff tried the bad tools like the default chatgpt 4o-mini (or tried the tools available 2 years ago) and got a bad experience. There are light-years of differences between these and something like o4-mini-high.

TL;DR: use the correct model for the job, and it doesn't really need to be an argument - if it makes you more productive it's a good tool, if it doesn't, nobody is forcing you to use it. But I don't think you should imply that everybody who likes these tools are stupid.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by ivape on Sunday June 01, 2025 @11:15AM (#44151498)
Stop telling everyone the secret.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by codr7 on Monday May 26, 2025 @11:52AM (#44098569)
Why?

What does it add that you couldn't have written yourself faster if you're so skilled?

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by SpaceNugget on Monday May 26, 2025 @01:00PM (#44099239)
> if you're so skilled?

I think this is needlessly snarky and also presupposes something that wasn't said. No one said it can write something that the developer couldn't write (faster) themselves. Tab complete and refactoring tools in your IDE/editor don't do anything you can't write on your own but it's hard to argue that they don't increase productivity.

I have only used cline for about a week, but honestly I find it useful in a (imo badly organized) codebase at work as an auto-grepper. Just asking it "Where does the check for X take place" where there's tons of inheritance and auto-constructor magic in a codebase I rarely touch, it does a pretty good job of showing me the flow of logic.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by cjaybo on Monday May 26, 2025 @01:14PM (#44099393)
Are these “millions of people” in the room with us now?
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by mattmanser on Tuesday May 27, 2025 @07:09PM (#44111448)
Where are these millions, and where is their output? You're in an echo chamber mate, there aren't millions of people using AI to do significant amounts of work.

Indie hackers just did an article on 4 vibe coded Startups and they all seem like a joke.

And they could only find 4!

I didn't look at them all, but the flight sim is spectacularly bad, the revenue numbers obviously unsustainable and it looks like something moderately motivated school children might have made for a school project in a week.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by l33tman on Friday May 30, 2025 @05:22AM (#44134407)
It's sort of a slow-motion avalanche. For example the price of outsourcing app development is really coming down now because it's one of the areas where the AI coding tools really excel. It's a lot of boiler-plate style code, pretty canonical stuff, no rocket science and, to be frank, not really that much that has to be clever. You can give the tools a screenshot of an UI and it gladly outputs correct styling code for it in seconds. It supercharges already skilled developers. And if you needed 6 in-house app developers before, now you only need 2. It isn't an immediate effect, but it is an effect that is slowly going to run though the business.

The question is, are companies going to use fewer people to do the same, or the same amount of people and just create better products?

For prototyping new ideas this is also an invaluable turbocharge for startups who can't really afford to have hordes of developers trying out alternative solutions.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by macNchz on Monday May 26, 2025 @11:38AM (#44098455)
Where I've found LLM coding assistants really effective in my client consulting work is around iteration: I can deliver so many more versions of an application UI in a given amount of time that it really changes the entire process of how we dig into a project. Where I might previously have wanted to start with low fidelity wireframes and go through approvals to avoid proto-duction and pain down the line when the client didn't like something, now we can rough out the whole thing in a functional proof of concept and then make sweeping changes live on a call as we test different interaction paradigms.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by ddingus on Monday May 26, 2025 @01:37PM (#44099645)
Your disclaimer adds value. Well done and appreciated!
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by deadeye on Monday May 26, 2025 @10:22AM (#44097690)
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by insane_dreamer on Monday May 26, 2025 @01:41PM (#44099673)
> create a React app with X amount of pages, some Redux stores, Auth, etc.

Unless you're a contractor making basic websites for small businesses, how many of these do you need to make? This really a small fraction of the job of most developers, except for entry-level devs.

> when it comes to maintaining existing systems, or adding more complicated features, or needing to know business domain details,

This is what experienced developers will spend 90% of their time doing.

So yes, LLMs can replace entry-level devs, but not the more experienced ones.

This begs the question: if companies stop hiring entry-level devs because LLMs can do their job, how will new devs get experience?

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by tkiolp4 on Monday May 26, 2025 @01:52PM (#44099791)
Your conclusion is wrong I think. LLMs cannot magically replace entry levels devs. Who’s gonna ask the LLM to create the basic website? The product owner? The accountant? The sales guy? They wouldn’t know how to be precise enough to state what they actually need. An entry level engineer would make use of the LLM to produce the website and push it to production. Hell, only engineers know that the devil is in the details. Quick example: let’s say a Contact Us page needs to be built. There are tons of details that need to be accounted for and the LLM may skip them if it is not told about them: where does the data of the form go to? A backend endpoint? What about captcha? What about analytics? What about validation of specific fields? What about the friendly URL? And disabling the button after sending to prevent duplicate requests?

An LLM is very capable of implementing all of that… if only someone who knows all of that stuff tell them first.

And most importantly: LLMs don’t challenge the task given. Engineers do. Many times, problems are solved without code.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by jimbokun on Monday May 26, 2025 @06:50PM (#44102369)
A lot of the value a good engineer provides is saying “you don’t want to do that, it’s a bad idea for these reasons.” Or “that’s actually easier than you think. We could do it this way.”

Knowing what’s possible, difficult, easy, risky, cheap, expensive, etc.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by insane_dreamer on Tuesday May 27, 2025 @09:38AM (#44106886)
> An LLM is very capable of implementing all of that… if only someone who knows all of that stuff tell them first.

I agree with you, but I don't think it's the entry-level dev who is going to be interfacing with the client to discuss and resolve all the questions you posed, and/or decide on them. That was part of the OP's point -- that much of their time is spent interfacing with the client to very precisely determine the requirements.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by elfly on Monday May 26, 2025 @04:36PM (#44101417)
yeah, this is the issue. I've used Claude Code to great success to start a project. Once the basic framework is in place, it becomes less and less useful. I think it cannot handle the big context of a full project.

It is something that future versions could fix, if the context a llm can handle grows and also if you could fix it so it could handle debugging itself. Right now it can do it for short burst and it is not bad at it, but it will get distracted quickly and do other things I did not ask for

One of these problems has a technical fix that is only limited by money; the other does not

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by jimbokun on Monday May 26, 2025 @06:42PM (#44102325)
To me that makes coding LLMs the new Excel.

I mean that in the sense that Excel is the tool that non developers could take the farthest to meet their needs without actually becoming a full time developer. But then to take it past that point a professional developer needs to step in.

I imagine non devs vibe coding their way to solutions far more complex than Excel can handle. But once they get past the greenfield vibe coding stage they will need a pro to maintain it, scale it, secure it, operationalize it, etc.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by prakashn27 on Monday May 26, 2025 @09:48PM (#44103229)
100% true.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by tmaly on Tuesday May 27, 2025 @03:01PM (#44109759)
I have not seen much discussion on how to properly work with legacy code using LLMs.

Michael Feathers book comes to mind when thinking about the topic. One gets the idea that you have to write a lot of tests. But what happens when there are no tests, comments, documents etc?

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by reg_dunlop on Saturday May 31, 2025 @10:01PM (#44148163)
> What I tend to do is write stubbed out code in the design I like, then I'll get an LLM to just fill in the gaps.

This seems like an interesting approach, though to me it begs the question: what does "stubbed out code" look like? How much stubbing is done? Have you considered using pseudocode as comments within a larger "stubbed out" portion?

The importance of rules and context has begun to elevate its significance (...that is, if context wasn't always very important), and finding ways to articulate that context seems to be a skill of greater importance...

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by loveparade on Monday May 26, 2025 @06:45AM (#44096058)
That's because the reality is somewhere in the middle. It obviously isn't 100x or even 10x except for very specific toy tasks, but for me it's probably around ~1.5-2x in everyday work. And I work on mostly long-tail scientific stuff, I imagine it's more if you do something like frontend dev with a lot of boilerplate. I'm absolutely convinced that people who say LLMs are not making them more productive either don't understand how to use them correctly, or they are working on very specific niche problems/infrastructure where they aren't useful.

2x may not sound like much compared to what you read in the media, but if a few years ago you had told companies that you can provably make their engineers even 2x more productive on average you'd probably be swimming in billions of dollars now. That kind of productivity increase is absolutely huge.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by tonyhart7 on Monday May 26, 2025 @10:34AM (#44097814)
"That kind of productivity increase is absolutely huge."

then company decide to hire less, the market flooded with junior AI powered skill, mid level programmer cant asking rise as easy anymore unless you are senior dealing with really specific stuff that need know how

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by codr7 on Monday May 26, 2025 @11:53AM (#44098584)
2x is also .5x in learning, nothing is free in this world.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by bluefirebrand on Monday May 26, 2025 @11:55AM (#44098593)
> I'm absolutely convinced that people who say LLMs are not making them more productive either don't understand how to use them correctly, or they are working on very specific niche problems/infrastructure where they aren't useful

I understand how to use LLMs, it's just a much worse workflow. Writing code isn't the hard part, reviewing code is much slower and painful process than writing it from scratch, when you actually know what you're doing.

I'm absolutely convinced that people who are saying LLMs are making them more productive either weren't very productive to begin with or they don't know what they're doing enough to realize what a bad job the LLM is doing

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by FiberBundle on Monday May 26, 2025 @04:30PM (#44101370)
Where's the actual evidence for the productivity boost though? Wouldn't one expect a huge increase in valuable software products or a dramatic increase in open source contributions if llms provide this kind of productivity increase?
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by mlsu on Monday May 26, 2025 @11:09PM (#44103636)
If it really is 2x in general, we would have noticed.

LLM are useful to me personally as a learning tool. It’s valuable to have it talk through something or explain why something is designed a certain way. So it’s definitely helped me grow as an engineer. But I have not come close to being 2x as productive, I don’t think my workload would be done by 2 people in the pre-LLM era. Maybe 1.25?

Because for the most part being a computer programmer is not about programming the computer but about… everything else.

Overall, LLM are a net negative for humanity in my opinion. The amount of slop on the internet now is absurd. I can never be sure if I’m reading the writing of a real person or a machine, which makes me feel like I’m slowly becoming brain poisoned by the devil himself on a psychological level. I have to think twice when I read any email longer than 2 paragraphs, or did my coworker start using more em dashes?

And also if it’s so damn easy for a machine to do all of this complex valuable work for a few kilojoules how complex or valuable is it anyway? Maybe it’s all just a fiction, that any of this stuff provides any value to anyone. Is another app really actually necessary? What about a hundred million new apps?! Even if we all became 100x as “productive,” what would that actually mean, in the real world you know the one that’s outside that has smells and textures and objects and people in it talking to one another face to face with their wet vocal chords??

These things are going to drive us all absolutely insane I can guarantee it. The logical end of this is complete madness.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by theappsecguy on Tuesday May 27, 2025 @09:57AM (#44107080)
It's really hard to measure. Sure, maybe 20-30% of the time I get a perfect autocomplete and boom, 2x speedup. Then there is the partially correct or completely incorrect suggestions that tools like Cursor keep spamming, making me either accept by mistake or continuously rage hit the escape key, which both slows things down and breaks the thinking flow.

How do you account for the latter? It's nearly impossible and I have seen no improvement in that realm. As the good suggestions have become better, the ratio of good to bad hasn't really changed for me.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by lm28469 on Monday May 26, 2025 @06:40AM (#44096017)
That's because they fall under the "engineers" category like my local postman and Michael Schumacher both fall under the "driver" category.

Most people don't work on anything technically hard, most problems are business logic issues that aren't solved technically or legacy code workarounds for which you need to put 3-10 domain experts in a room for a few hours to solve.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by sunrunner on Monday May 26, 2025 @07:19AM (#44096253)
There are a lot of terms in software development that have been co-opted from other disciplines and misrepresent a lot of development work, including 'engineering' and 'architecture'.

I think it's helpful to think of engineering as a _process_ instead of a role, and the reality is that a lot of development work doesn't necessarily rely on the strong engineering methodology (e.g. measurement, material properties, tolerances, modelling, etc.) that the people developing the software might imagine just based on the number of job adverts for 'engineers'.

This isn't a bad thing. There are hundreds or thousands of different but equally valid solutions to getting a program to do a thing, but not recognising that most code writing is somewhere between a art and engineering and is neither a purely artistic discipline but also rarely a purely engineering one is useful.

The kinds of engineering and architecture that people think of in software development only really represent common practices and shared language (e.g. design patterns, architectural patterns) and not a strong engineering practice or any kind of truth about how software actually runs.

(Unless you're writing software for launching rockets, in which case the engineering _process_ porbably should be strong).

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by cassianoleal on Monday May 26, 2025 @05:52PM (#44102018)
> the reality is that a lot of development work doesn't necessarily rely on the strong engineering methodology (e.g. measurement, material properties, tolerances, modelling, etc.)

It's probably true that a lot of development work doesn't rely on those. It's probably also true that the work other kinds of engineers do also don't.

That said, when engineering software systems, those are very important. Measurement: resource sizing, observability; tolerances: backoffs, back pressure, queues, topics, buffers; modelling: types, syntax, data analytics...

There's a whole class of developers out there that are not aware or very good at those. And that's fine. There's a place for them in the market. You don't need an engineer to work on your floor joists or your plumbing. Sure you can have one, but you can also hire a builder or DIY it all yourself.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by nobodywillobsrv on Monday May 26, 2025 @07:07AM (#44096184)
So true and it makes me feel better to have this mental model. I find it useful for simple things that are perhaps just beyond search and copy paste but sometimes wonder if the more technical stuff is not even quite right as it is possible to sort of brainwash yourself into not thinking clearly when nagging an llm into being reasonable on something it has a hard time with.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by voidhorse on Monday May 26, 2025 @09:34AM (#44097307)
Completely agree. If anything LLMs have just made this completely transparent and are revealing that actual engineering in software is limited to rare cases.

All of theses middling developers who are so excited about these tools don't seem to realize that they are perfectly primed to eventually eliminate precisely the kind of worker responsible for hooking up existing APIs for solved problems into a web app, the kind of work the market hitherto greatly over esteemed and overpaid for isn't my going to be so esteemed or highly valued anymore. The only real work that will remain is actual hard engineering work (solving novel technical modeling problems, not just plugging apis together). All of these lazy devs are hyped for precisely the automation that's goin. to significantly reduce their pay and labor prospects in the long term lol. I'm shocked at times at how people can fail to see the writing on the wall when it's been written in gargantuan red ink

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by esafak on Monday May 26, 2025 @02:39PM (#44100249)
Software engineering work never ends, so they'll just work faster.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by mbac32768 on Monday May 26, 2025 @09:57AM (#44097468)
Right yes, I imagine most people who program are in companies that have nothing to do with high tech, writing SQL or scripts or CRUD apps. LLMs are probably amazing here.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by t917910 on Monday May 26, 2025 @09:57PM (#44103278)
Writing SQL in the context of 20 years old application with 2500 stored procedures just inside one of many databases is not so easy for LLM.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by SatvikBeri on Monday May 26, 2025 @06:43AM (#44096044)
I find LLMs make me about 10% more productive – which is great, but nowhere near as extreme as some claims.

Cursor autocompletes a lot of stuff I would have typed anyway, and LLM search is a strong complement to Google, though not a direct replacement. Generating more than a few lines usually gives me bad code, but often suggests libraries I didn't know about, especially outside my area of expertise.

There's just not a lot of people talking about middling results because the extremes tend to eat up the conversation.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by codr7 on Monday May 26, 2025 @11:58AM (#44098628)
It's not a free lunch though, you're sacrificing reinforcement learning and becoming more dependent on your tools.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by jimbokun on Monday May 26, 2025 @07:01PM (#44102404)
I do find AI a replacement to Google most of the time, because now when I search on Google the Gemini results at the top generally give me what I wanted to know.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by melvinroest on Monday May 26, 2025 @07:05AM (#44096174)
I'm learning math through Math Academy, ChatGPT and YouTube (mostly 3Blue1Brown and similar channels). Without that specifc combination, it would've been hell. Now it's just nice.

A few years I did it with something similar to Math Academy (from the Universiteit of Amsterdam). ChatGPT wasn't intelligent enough back then so I didn't use it. It felt a lot tougher.

ChatGPT answers questions that a teacher would find obscure nonsense, but I'm the type of student that needs to know the culture behind math so I can empathize with the people doing it and can think of solutions via that way. Like why is the letter m used when talked about an angle? Mathematically, it shouldn't matter. But it irritates me as I'd use a instead, it makes more sense as you want to lower the cognitive burden and focus as much on the math as possible. So I ask ChatGPT and one of the things it said is that historically it meant "measure". Now, I get it and can focus on the actual math again.

Another example is the fast variance calculation: average_of_squares - square_of_the_average

how does that come from (1/n) * sum((x - mean)^2) ?

It just shows it.

World domination? Nope, but I'm learning something I otherwise wouldn't. But as you can tell, YouTube and Math Academy play their own roll in it.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by elzbardico on Monday May 26, 2025 @07:19AM (#44096257)
You'd love a very old book named Mathematics for the Million. The author was a committed Marxist and an Engineer, and due to this combination, it places a strong emphasis on demystifying concepts, the historical context of mathematical ideas and its practical application.

Of course, some of it is a bit strange for modern audience, the way lots of things in calculus are taught nowadays is completely different. If you are looking for rigorous formal proofs, you won't find them on this book.

But it is a quite funny and quirk book that gives you a different perspective on math concepts and will satisfy your hunger for knowing the people and the historical context behind it.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by namaria on Monday May 26, 2025 @07:16AM (#44096234)
> So I ask ChatGPT and one of the things it said is that historically it meant "measure"

This is a great example of why using LLMs to learn is perilous. This is nonsense.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by melvinroest on Monday May 26, 2025 @07:25AM (#44096319)
Is it? I didn't bother to check as it wasn't too relevant. I wasn't stuck anymore. Worst-case it's a memnonic that is untrue. Best case: it is. For actual derivations I check the work.

This is what it wrote by the way.

I posted an image and asked:

Why is this m

m for angle mangle?

ChatGPT:

In geometry we often need to distinguish the angle itself (a geometric object) from the numerical size of that angle (a real-number measurement, usually in degrees or radians). To keep the two ideas separate we put a little “m” in front of the angle symbol:

m∠B reads as “the measure of angle B.”

* ∠B refers to the actual wedge-shaped region determined by the two rays that meet at vertex .

* m∠B is the single number—e.g. \90∘—that tells you how “wide” that wedge is.

So in the worked example the author writes

m∠A + m∠B + m∠C + m∠D = 360∘

because they are adding the sizes of the four interior angles of a quadrilateral, not the angles themselves.

If you dropped the m you’d be adding geometric objects, which doesn’t make sense; the m reminds the reader that what’s being added are real-number measures. (Some textbooks use ∣∠B∣ or simply write the letter with a bar over it, but “m∠” is by far the most common in U.S. and many international curricula.)

So the mysterious “m” isn’t another variable—just a shorthand for “measure of.”

---

Edit: this took quite some work to write well as ChatGPT answers don't copy/paste neatly into HN. So I won't be doing this for the other example.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by card_zero on Monday May 26, 2025 @07:45AM (#44096463)
See slope: https://en.wikipedia.org/wiki/Slope

It isn't customarily used for angles (those get Greek letters).

The m stands for mystery.

Edit: ah, but I see that this prefixed m for "measure" is also used sometimes. It appears at https://en.wikipedia.org/wiki/Angle#Combining_angle_pairs though I'm not sure why it's necessary. Maybe because you want to clarify absolute magnitude is meant, to avoid adding negative values.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by evertedsphere on Monday May 26, 2025 @08:22AM (#44096750)
yes, it's the equivalent of |AB| for a line segment and is not uncommon in high school maths texts in some parts of the world
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by namaria on Monday May 26, 2025 @09:03AM (#44097080)
I thought it was a different kind of nonsense, but it still has a subtle error. Goes to show even more how risky it is to repeat LLM answers as factual.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by melvinroest on Monday May 26, 2025 @10:27AM (#44097739)
Could you point out what you mean? It's really hard to follow you. You say it's nonsense but it is not clear why. Then I write down a response that took me quite a while to format correctly (and to look up) and you then say "I thought it was a different kind of nonsense." Then you say it still has a subtle error.

It is really hard to follow you if you don't explain yourself.

I'm not saying it's factual. The reason I showed that answer was simply to verify to see if it was what you thought it was (hence I asked "is it?"). It turns out that it wasn't fully.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by ants_everywhere on Monday May 26, 2025 @10:05AM (#44097535)
ChatGPT is right, although I'm not sure how historical the notation is.

∠ is traditionally a function from points to axiomatic geometric objects. ∠ABC is the angle at B oriented so that we start at A, go to B, then to C.

Your text seems to be using ∠ either as a kind of type annotation (indicating by ∠B that B is an angle) or (perhaps more likely) is just suppressing the other letters in the triangle and is short for something like ∠ABC.

Since ∠B is an axiomatic Euclidean object, it has no particular relation to the real numbers. m is an operator or function that maps axiomatic angles to real numbers in such a way that the calculations with real numbers provide a model for the Euclidean geometry. Why call it m? I'm not aware of it being historical, but almost certainly it comes from measure, like the μ in measure theory.

Obviously ∠ is a graphical depiction of an angle, and my guess is it probably evolved as a shorthand from the more explicit diagrams in Euclid.

Traditionally angles are named with variables from the beginning of the Greek alphabet: α, β, γ. Then we skip to θ presumably to avoid the Greek letters that look nearly identical to Roman letters.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by melvinroest on Monday May 26, 2025 @10:21AM (#44097684)
"I'm not sure how historical the notation is."

I conflated this with another ChatGPT conversation where it gave 3 possible historical sources for another symbol that I fell over and then had trouble proceeding.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by wolfhumble on Monday May 26, 2025 @09:15AM (#44097149)
> This is a great example of why using LLMs to learn is perilous. This is nonsense.

These type of answers from teachers, co-students, web communities, blogs etc. are – I would assume – why people ask LLMs in the first place.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by namaria on Monday May 26, 2025 @09:32AM (#44097297)
It is a problem that people who are unwilling to perform some basic research resort to 'learning' the output of LLMs. No one is entitled to answers.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by melvinroest on Monday May 26, 2025 @10:35AM (#44097839)
"No one is entitled to answers" feels very definitive, defeating and tiring. Especially because you don't explain your own thought process.

Could you please assume a good faith discussion?

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by namaria on Tuesday May 27, 2025 @04:33AM (#44105020)
I have. I was criticized for pointing out spurious nonsense in LLM slop by someone who claimed people wouldn't have to resort to it if other people made an effort to explain things better.

But I don't believe anyone is entitled to an explanation. I find things out by looking up books and testing things. Any explanation someone deigns to give me is a bonus and doubted until corroborated.

I don't know why anyone would think they are owed a custom explanation for their specific questions and thinking like that will get you in trouble when you come to depend on what anyone (or anything) is willing to chew up for you.

Maybe I was terse but I don't think I was rude or illogical.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by melvinroest on Tuesday May 27, 2025 @10:19AM (#44107268)
> I was criticized for pointing out spurious nonsense in LLM slop

I can see that you experience it as such but I think it's more of a spectrum. Often times, LLMs give good answers. Often enough times they don't. One needs to keep that in mind. In my example, given that it was just a symbol, all I needed was knowledge at the level of a memnonic which would, on average, at least somewhat also point directionally to the truth. But that's a bonus. I could make up a memnonic myself, but I like having that bonus.

Given that ChatGPT is directionally towards the truth, but not fully (on average), I'd need to test it or verify it if I want a better level of knowledge than that. If that's the case, then ChatGPT is basically acts as a sort of cache as it's quicker to ask a question to ChatGPT than to research on one's own. One can experience a cache hit or cache miss. Such a thing will happen in the verification stage. Specifically, for math this is quicker, in my experience.

But anyways that's my experience. Your experience is that it's spurious nonsense slop. And I suppose you therefore find it a problem. I don't see the issue as there are different levels of knowledge and different time commitments you need to give to them. A lot of my knowledge is based on trust anyway and sometimes it's broken (e.g. replication crisis in psychology, I felt betrayed having studied the field).

> I don't know why anyone would think they are owed a custom explanation for their specific questions

I'm not sure if anyone said anything like it. Regardless of that, the need still exists. People will still act on that need. I suspect you see that as a problem. I'm neutral on it.

> thinking like that will get you in trouble when you come to depend on what anyone (or anything) is willing to chew up for you.

IMO teaching and learning is a 2 way street. It's the teacher's job to explain it well enough. It's the student's job to do their best to understand it. Math Academy offers exercises and explanations. Sometimes I find their explanations a bit lacking. So I use other sources to augment it.

> Maybe I was terse but I don't think I was rude or illogical.

Reading/writing text is tough, which is why I stated how I felt. It'd probably have been easier in an actual conversation. I didn't mean to imply you were being rude or illogical.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by layer8 on Monday May 26, 2025 @12:46PM (#44099092)
> why is the letter m used when talked about an angle?

This isn’t the best example, because a simple Google search gives you immediate results for that as well, and arguably more nuanced ones. You’ll learn, for example, that m is also the standard letter for slope, which may or may not be related.

Also, if a teacher finds that question “obscure nonsense”, I’d argue that they are a bad teacher.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by melvinroest on Tuesday May 27, 2025 @10:03AM (#44107138)
Hmm, it's not the best example yea. It's just that ChatGPT understands math notation a lot better than Google. If needed, you can always let ChatGPT do a Bing search.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by infecto on Monday May 26, 2025 @07:25AM (#44096318)
You’ll usually find three kinds of people talking about LLMs online:

1. The ones writing Medium posts about being 100x more productive. 2. The skeptics, like yourself, who construct made-up examples to prove it’s impossible to get value, framing things in a way where failure is the only plausible outcome. 3. And then there’s the middle group, people quietly getting real, uneven, but worthwhile value.

You’d be surprised. The vast majority of engineering isn’t that unique. Usefulness often depends on the age of the codebase and the dynamics of the team. I’m firmly in the middle camp: far from perfect, but I get real value from these tools as they stand today. It’s like having a junior engineer who can scaffold out the basics so I can spend my mental energy on the parts that actually matter.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by soraminazuki on Monday May 26, 2025 @07:53AM (#44096520)
GP didn't "construct made-up examples" anywhere in this thread as of the time of writing this post. Where did you hallucinate that from?

Seriously, why is every comment hyping up AI generated code like this. No concrete evidence, just turtles all the way down. But when actual examples do show up, it's a complete clown show [1][2].

[1]: https://news.ycombinator.com/item?id=44050152

[2]: https://news.ycombinator.com/item?id=43907376

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by infecto on Tuesday May 27, 2025 @06:38AM (#44105610)
He definitely crafted the narrative he wanted, you see that just as much from skeptics as from the hype crowd. I still stand by what I said: there are always three camps in any LLM discussion. No hallucinations here, I can see where you land. I’m here for a constructive argument, and I believe there’s a real middle ground where LLMs are creating tangible value, even for experienced engineers.

Your links about GitHub agents may be spot-on, I haven’t used them myself, so no strong opinion there. Same with Codex; it’s early. But why not mention tools like Cursor or Windsurf? Or the open source alternatives. That’s where people are seeing actual gains today. Why go all-in on anti-hype without talking about your direct experience?

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by soraminazuki on Friday May 30, 2025 @03:44AM (#44133902)
So you shifted from "constructing made-up examples" to "crafting narratives." In plainer terms, your accusation downgraded from calling someone a liar to merely suggesting that they are making an argument. Is the latter accusation supposed to be a bad thing in discussions?

Anyways if you want to hear my direct experience, I reviewed AI generated code. It couldn't get a simple map, reduce, and filter right. I personally wouldn't waste my coworker's time with such slop. I'm happier coding with existing tools for boilerplate generation and code completion, along with my own brain. Much more predictable and reliable.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by bobxmax on Monday May 26, 2025 @06:48AM (#44096071)
They are.

People are building sophisticated products in days not months now. If you can't see that, it's your issue.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by gyomu on Monday May 26, 2025 @07:04AM (#44096167)
Would you be so kind as to link to some of these sophisticated products? Would love to check them out.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by bobxmax on Monday May 26, 2025 @07:32AM (#44096381)
Go look at the latest YC batch. 30% of them were built with 90% of the code being written by LLMs.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by Jensson on Monday May 26, 2025 @08:09AM (#44096661)
Where did you see that stat? And why just 30%, shouldn't almost all of them be built by LLM if LLM is that much better?

Edit: This says a quarter, where did you get the 30% figure from?

https://techcrunch.com/2025/03/06/a-quarter-of-startups-in-y...

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by bookofjoe on Monday May 26, 2025 @08:53AM (#44097003)
Are you seriously pointing out the difference between 25% and 30% as meaningful?
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by bobxmax on Tuesday May 27, 2025 @11:54AM (#44108160)
wow 25% not 30% ya sure got me

> shouldn't almost all of them be built by LLM if LLM is that much better

a) not every yc company is a software company

b) these things take time, and not everyone knows how to use these tools

this stuff should be pretty self evident.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by viridian on Wednesday May 28, 2025 @06:55AM (#44114654)
Perhaps bobxmax should have asked an LLM what the exact number was instead of relying on his outdated & error prone human memory.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by codr7 on Monday May 26, 2025 @12:02PM (#44098678)
Yeah, let's all pull numbers out of a hat and see who wins.

Learn some real skills or prepare to not have a job.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by bobxmax on Tuesday May 27, 2025 @11:54AM (#44108157)
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by codr7 on Monday May 26, 2025 @12:01PM (#44098666)
Good luck with what, truth will find you and keep you accountable.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by traceroute66 on Monday May 26, 2025 @09:17AM (#44097168)
> And those who, like you, find it to be an extremely finicky process that requires extreme amount of coddling to get average results at best.

I am very much in that category.

I would describe my experience as the old adage "why keep a dog and bark yourself".

And no, I don't buy into people who say you have to engage in "prompt engineering". That's bullshit. Let's face it "prompt engineering" is a synonym for "I have to waste time coming up with hacky workarounds to make this dumb LLM come up with something usable".

In the end, its always quicker and easier to do it myself. And unless you are fresh out of school, if you've got more than 10 minutes of coding experience, you will always do a better job than the LLM.

Every time a new LLM model comes out, I fall into the same old trap "oh well, this one must be better, surely" I say to myself.

And yet its always the same "you can't polish a turd but you can roll it in glitter" experience.

I'm sick and tired of LLMs hallucinating shit.

I'm sick and tired of LLMs inventing functions that have been available in stdlib for years.

I'm sick and tired of LLMs generating useless code with boilerplate subfunctions that just contain a single commented "implement it yourself" line.

On top of that LLMs are simply terrible for the environment, guzzling up all that electricity, producing nothing but hot air and bullshit results in return.

Basically I'm sick and tired of LLMs and all the LLM fetishists that surround them.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by exe34 on Monday May 26, 2025 @09:34AM (#44097305)
I've managed to use ChatGPT to write a small app in Kotlin, and later used Cursor to fix some issues around performance. Personally I'm not a finish-it kind of guy unless I'm getting paid for it.

To be fair, I'm not sure the quality of the code is much worse than all the boatloads of crapware in all the app stores out there.

I suspect that the reason "revolutionary products" and "blazing fast iteration" isn't blowing our minds is because the problem was never the code - it was always the people around it, who know better than their customers. The best way to make good software is to test it on real people and iterate - that's going to take a long time no matter how fast the coding part is.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by dogtierstatus on Monday May 26, 2025 @09:41AM (#44097353)
This is a quality vs quantity issue. There were x number of extremely competent and ~10x moderately competent programmers.

LLMs have only made this to 100x moderately competent programmers and may be 2xed extremely competent ones?

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by analog31 on Monday May 26, 2025 @10:05AM (#44097534)
Even 2x would be an improvement. It would mean that a project goes from being 3 years late, to only 1.5 years late.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by binary132 on Monday May 26, 2025 @10:04AM (#44097522)
I think a big part of the problem is the devx / workflows that these tools tend to encourage or impose. Watching my coworkers fumble with terrible copilot autoshit in a pairing session makes me want to scream on a daily basis. That doesn’t necessarily mean such a DX is the only way it could or should work, or that it’s even a good DX at all. Most of the time, it actually imposes more cost, both in lost momentum and in corrective action, than it relieves.

I think something like a graph where nodes are specified clearly with inputs and outputs (or pre- and post-conditions, if you prefer), and agents iteratively generate and refine the implementation details could have some utility, but that would require programmers to know how to think about program design (and have a usable UI/UX for such a tool) instead of hastily stumbling through patches and modifications to existing code they already don’t understand.

The new LLM systems are carrying us farther from that world, not closer to it.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by wooque on Monday May 26, 2025 @10:28AM (#44097753)
Because code is small part of making successful business and usually not the bottleneck.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by michaelrpeskin on Monday May 26, 2025 @10:31AM (#44097786)
A little snarky but: In my experience, the folks who are 100x more productive are multiplying 100 times a small number.

I've found great success with LLMs in the research phase of coding. Last week I needed to write some domain-specific linear algebra and because of some other restrictions, I couldn't just pull in LAPACK. So I had to hand code the work (yes, I know you shouldn't hand code this kind of stuff, but it was a small slice and the domain didn't require the fully-optimized LAPACK stuff). I used an LLM to do the research part that I normally would have had to resort to a couple of math texts to fully understand. So in that case it did make me 100x more effective because it found what I needed and summarized it so that I could convert it to code really quickly.

For the fun of it, I did ask the LLM to generate the code for me too, and it made very subtle mistakes that wouldn't have been obvious unless you were already an expert in the field. I could see how a junior engineer would have been impressed by it and probably just check it in and go on.

I'm still a firm believer in understanding every bit of code you check in, so even if LLMs get really good, the "code writing" part of my work probably won't ever get faster. But for figuring out what code to write - I think LLMs will make people much faster. The research and summarize part is amazing.

The real value in the world is synthesis and novel ideas. And maybe I'm a luddite, but I still think that takes human creativity. LLMs will be a critical support structure, but I'm not sold on them actually writing high-value code.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by data-ottawa on Monday May 26, 2025 @11:31AM (#44098401)
They definitely help raise the floor, but have a much lower ceiling.

If you’re near that ceiling you get most value out of code reviews and those side features you don’t care about that allow you to focus more on the actual problem.

That’s a 10-20% lift instead of 10/100x.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by ryandrake on Monday May 26, 2025 @12:37PM (#44099007)
To steal the joke from that vintage coffee tin: LLMs help you "do stupid things faster." If all you want to do is dump terrible code into the world as fast as possible, then you've got a great tool there. If you're looking to slowly and carefully build a quality product on a solid foundation, or create a programming work of art, LLMs are at best a marginally better autocomplete.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by Brystephor on Monday May 26, 2025 @12:42PM (#44099058)
> I've found great success with LLMs in the research phase of coding.

This is what I've found it most helpful for. Typically I want an example specific to my scenario and use an LLM to generate the scenario that I ask questions about. It helps me go from understanding a process at a high level, to learning more about what components are involved at a lower level which let's me then go do more research on those components elsewhere.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by jghn on Monday May 26, 2025 @10:34AM (#44097822)
> There are two kinds of engineers.

Apparently there are at least three as I fit neither of these molds of yours. They are neither making me 100x more productive, nor am I putting in an extreme amount of coddling.

For context, in the 30ish years I've been writing code professionally, I've *always* needed to look stuff up constantly. I know what I want, and I know what's possible. I just can never remember things like syntax, specifics, that sort of thing. It doesn't help that I often wind up in jobs where I'm switching between languages often.

Before quality search engines like Google, this meant I was always flipping through reference books, man pages, and worse. Then came search engines and that was a step more productive. Then came things like SO, and that was a step more productive. I find LLMs to be another step forward, but not like a quantum leap or anything. Take autocomplete suggestions for instance: Sometimes it's exactly the prompt (pardon the pun) that I need to jog my memory. Other times I know it's not what i want and I ignore it. And the chat interfaces I find better than Googling SO as I can have a back & forth with it.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by mattfca on Monday May 26, 2025 @04:40PM (#44101461)
this describes me and my experience exactly
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by kordlessagain on Monday May 26, 2025 @10:40AM (#44097884)
“If we do not invent the future, someone else will do it for us - and it will be their future, not ours.” – Duncan D. Bruce & Dr. Geoff Crook, The Dream Café
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by GenerocUsername on Monday May 26, 2025 @10:56AM (#44098052)
At a macro level, people from the former group might be doing better than you think.

I have gone from Senior II, to Lead Architect, to Principal Engineer in the years LLMs have been available.

It's not literally a 100x multiplier, but I have leveraged their ability to both fill in details where I lack specific knowledge, and as a first point of contact when researching a new space.

LLMs have enabled me to widen my sphere of influence by deepening my abilities at every point of my T-shaped knowledge curve.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by codr7 on Monday May 26, 2025 @11:55AM (#44098603)
Except you're not deepening anything, rather becoming increasingly dependent on wacky tools to keep up your fake status. Go for it, but it's a recipe for disaster imo.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by conductr on Monday May 26, 2025 @11:11AM (#44098211)
My guess is it depends on what you work with. I’ve been impressed by my ability to spin up nice looking websites and landing pages. Yet, I’ve tried to build a game and despite it appearing to work, it eventually has gotten confused about its own codebase and I end up with a ball of spaghetti where requirements of yesterday get lost or broken when adding functionality today. It feels more like I’m chasing my tail, vibe coding as they say, it’s just seems endlessly circular.

My game dev is going to be a hybrid approach going forward. I’ll ask it to create a new project with the desired functionality and see how the solution was proposed and then incorporate that into my main code branch.

I find this easier than the Cursor roll back or being overly cautious with suggested edits. I’m just getting to this realization so tbd if it works well. FYI, I’m good at programming generally but game dev is my Achilles. We’ll see how it it goes.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by peacebeard on Monday May 26, 2025 @11:11AM (#44098214)
I don’t fit in either category. My experience is that LLMs are good at writing code that is easy to write. This is not game-changing, but it is useful.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by emp17344 on Monday May 26, 2025 @11:30AM (#44098395)
The fact that no one can agree on how useful these things are makes me think they aren’t that useful.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by slightwinder on Monday May 26, 2025 @11:34AM (#44098419)
Even if someone seriously works 100 times faster with AI, it doesn't mean they can easily dominate any established market. A product is more than just code, and a company is more than their headcount. There is domain knowledge about the product and market. There is the personal ability of the workers and the synergy coming with, also the strength of established structures grown over years and decades. Just the habit and trust of users is a big fortress every new product has to conquer. That's not something some random dev or team of devs can easily reach just by being 100 times faster.

And realistically, there is also the question of where this claim is coming from. Someone being 100 times faster with AI is probably not starting from a place of high competence. Of the 100x, 90x are probably just for filling the gap to peak devs. So at the end they would be probably as productive as a highly competent dev of some domain, just that they can more easily access any domain without years of experience in that domain. And if we are honest, we somewhat have that already without AI, just by people copying and cookie cutting any sh*t

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by max_on_hn on Monday May 26, 2025 @12:09PM (#44098743)
(disclaimer: I have a vested interest in the space as the purveyor of an AI software development agent)

The result you described is coming soon. CheepCode[0] agents already produce working code in a satisfying percentage of cases, and I am at most 3 months away from it producing end-to-end apps and complex changes that are at least human-quality. It would take way less if I got funded to work on it full time.

Given that I'm this close as a solo founder with no employees, you can imagine what's cooking inside large companies.

[0] My product, cloud-based headless coding agents that connect directly to Linear, accept tickets, and submit GitHub PRs

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by johnecheck on Monday May 26, 2025 @01:38PM (#44099648)
"At most 3 months away"

This is a strong claim. I'm not aware of any AI model or system that can consistently and correctly make complex changes in existing codebases. What makes you so confident that you can achieve this, alone, in the next 3 months?

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by t917910 on Monday May 26, 2025 @10:04PM (#44103308)
The circumference of a circle is easily calculated as piradius2, so calculating the same length of an ellipse would not be much more difficult. Hehe.
Re:Ask HN: Anyone struggling to get value out of coding LLMs? by sanderjd on Monday May 26, 2025 @01:10PM (#44099343)
Nah, there's a third kind, which is probably the quieter majority. These are people who find these to be useful but imperfect tools, whose expectations remain realistic, and for whom the continuing progress these tools are making is pleasant and not exactly surprising but also not taken as a given.

I'm finding that these tools are supporting an ever expanding portion of my work, while remaining frustrating frequently.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by tom_m on Monday May 26, 2025 @02:49PM (#44100370)
It's a good sounding board and productivity tool...but it's over hyped.

I think it'll calm down a bit. People are just making huge claims right now which is music to the ears of many.

I like the tools don't get me wrong, but they aren't going to make a huge difference day to day. The area they will make a huge difference is in helping people work out issues faster. You know those multi-hour or multi-day issues you run into where you go crazy reading and re-reading the code. The LLMs are good at giving you a fresh look at the code that you'd otherwise need to get more people to help with or go sleep on it for a day.

This time savings here is probably the most significant time savings AI code editors or agents will ever provide.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by gofreddygo on Tuesday May 27, 2025 @02:53AM (#44104524)
There's a third kind that agrees these are stupid but strongly believes, beyond doubt, they'll be 100x better in 2 years.

Everything AI generated today is hot garbage but hey, just wait 2 years and the road to eldorado will show itself.

pfft.

Re:Ask HN: Anyone struggling to get value out of coding LLMs? by jf22 on Wednesday May 28, 2025 @11:48AM (#44117308)
There is a gradient in here. The difference between the two isn't that extreme.