News for hackers. Stuff that matters.
Re:With AI You Need to Think Much Bigger by xrd on Monday March 10, 2025 @09:51AM (#43320610)
Where are the LLM leaderboards for software estimation accuracy?

I have been using Claude Code and Aider and I do think they provide incredibly exciting potential. I can spin up new projects with mind boggling results. And, I can start projects in domains where I previously had almost no experience. It is truly exciting.

AND...

The thing I worry most about is that now non-technical managers can go into Claude and say "build me XYZ!" and the AI will build a passable first version. But, any experienced software person knows that the true cost of software is in the maintenance. That's 90% of the cost. Reducing that starting point to zero cost only reduces the total cost of software by 10%, but people are making it seem like you no longer need a software engineer that understands complex systems. Or, maybe that is just my fears vocalized.

I have seen LLMs dig into old and complex codebases and fix things that I was not expecting them to handle. I tend to write a lot of tests in my code, so I can see that the tests pass and the code compiles. The barbarians have come over the walls, truly.

But, IMHO, there is still a space where we cannot ask any of the AI coding tools to review a spec and say "Will you get this done in 2 months?" I don't think we are there, yet. I don't see that the context window is big enough to jam entire codebases inside it, and I don't yet see that these tools can anticipate a project growing into hundreds of files and managing the interdepedencies between them. They are good at writing tests, and I am impressed by that, so there is a pathway. I'm excited to understand more about how aider creates a map of the repository and effectively compresses the information in ways similar to how I keep track of high level ideas.

But, it still feels very early and there are gaps. And, I think we are in for a rude awakening when people start thinking this pushes the cost and complexity of software projects to zero. I love the potential for AI coding, but it feels like it is dangerously marketing and sales driven right now and full of hype.

Re:With AI You Need to Think Much Bigger by mentalgear on Monday March 10, 2025 @10:27AM (#43321042)
The "coding benchmarks" like "SWE-verified" are actually of very low quality and the answer riddled with problems.

Good Explainer: "The Disturbing Reality of AI Coding" https://www.youtube.com/watch?v=QnOc_kKKuac

Re:With AI You Need to Think Much Bigger by rurp on Monday March 10, 2025 @10:55AM (#43321366)
This largely fits with a pattern I've been seeing with LLM coding. The models are often helpful, sometimes extremely so, when it comes to creating prototypes or other small greenfield projects. They can also be great at producing a snippet of code in an unfamiliar framework or language. But when it comes to modifying a large, messy, complicated code base they are much less helpful. Some people find them a useful as a beefed up autocomplete, while others don't see enough gains to offset the time/attention to use them.

I think a lot of arguments about LLM coding ability stem from people using them for the former or the latter and having very different experiences.

Re:With AI You Need to Think Much Bigger by aaalll on Monday March 10, 2025 @11:53AM (#43321884)
There is a tool cool lovable that is basically targeted at that exact thing having designers and product managers get something that kinda works
Re:With AI You Need to Think Much Bigger by someothherguyy on Monday March 10, 2025 @10:18AM (#43320928)
> I can spin up new projects with mind boggling results

Boggle a skeptical mind

Re:With AI You Need to Think Much Bigger by Workaccount2 on Monday March 10, 2025 @11:30AM (#43321668)
I used claude to create a piece of software that can render gerber files (essentially vector files used in electronics manufacturing), overlay another gerber file on top of it with exact alignment, and then provide a GUI for manually highlighting components. The program then calculates the centroid, the rotation, and prompts for a component designator. This all then gets stored in a properly formatted Place file, which is used for automating assembly.

The day before this I was quote $1000/yr/user for software that could do this for us.

Re:With AI You Need to Think Much Bigger by xrd on Monday March 10, 2025 @10:37AM (#43321151)
Meaning, give you an example?

This morning I created a new project. I provided a postgres database URL to a remote service (with a non-standard connection string, includes a parameter "?sslmode=require"). Then, I said:

  * "Write me a fastapi project to connect to a postgres database using a database url."
  * "Retrieve the schema from the remote database." It used psql to connect, retrieves the schema. That was unexpected, it figured out not only a coding task, but an external tool to connect to a database and did it without anything more than me providing the DATABASE_URL. Actually, I should say, I told it to look inside the .env file, and it did that. I had that URL wrong initially, so I told it to reload once I corrected it. It never got confused by my disorganization. 
  * It automatically added sqlalchemy models and uses pydantic once it figured out the schema.
  * "Create a webpage that lets me review one table."
  * "Rewrite to use tailwindcss." It adds the correct tailwindcss CDN imports.
  * It automatically adds a modal dialog when I click on one of the records.
  * It categorized fields in the database into groupings inside the modal, groupings that do indeed make sense.
I know the devil is in the details, or in the future. I'm sure there are gaping security holes.

But, this saved me a lot of time and it works.

Re:With AI You Need to Think Much Bigger by xrd on Monday March 10, 2025 @02:12PM (#43323678)
And, the update is that in the last hour claude somehow removed the rendering of the actual items. It is clever in troubleshooting: it created mocks if the data could not be loaded, it added error messages with the number of items retrieved. But, it does not have access to my browser console nor the DOM, and therein lies the problem. It is slow to copy and paste back and forth from the browser into the terminal. But, this feels like a great opportunity for a browser extension.

But, my takeaway is that I could have fixed this exact problem in five minutes if I had written the JS code. But, I have not looked at anything other than glancing at the diffs that fly by in the session.

Re:With AI You Need to Think Much Bigger by xur17 on Monday March 10, 2025 @01:52PM (#43323279)
I used Claude + aider to create a webhook forwarding service (receives webhooks, forwards them to multiple destinations, and handles retries, errors, etc). Included a web interface for creating new endpoints and monitoring errors, and made it fully dockerized and ready to be deployed into a kubernetes cluster.

Took me like 3 hours and a few dollars in api credits to build something that would have taken me multiple days on my own. Since it's just an internal tool for our dev environments that already does what we need, I don't care that much about maintainability (especially if it takes 3 hours to build from scratch). That said, the code is reasonably usable (there was one small thing it got stuck on at the end that I assisted with).

Re:With AI You Need to Think Much Bigger by xrd on Monday March 10, 2025 @02:13PM (#43323687)
Claude Code and Aider, or aider using claude sonnet as the model? If you are using both claude code and aider I would love to know why.
Re:With AI You Need to Think Much Bigger by xur17 on Monday March 10, 2025 @02:16PM (#43323761)
Aider + Claude sonnet as the model.
Re:With AI You Need to Think Much Bigger by brulard on Monday March 10, 2025 @05:20PM (#43326232)
I wanted a whiteboard app - simple, infinite, zoomable, scrollable svg-based area with custom shapes and elements. It just needed prompts like "add zoom", "add a rect shape", "add control points for resize", "add text shape", "make position attributes able to reference another objects properties, + basic arithmetic" (for example to make line connect 2 objects, or size of one mirror size of another, etc.). It made all of these with so little effort from me. I would never undertake such a project without an LLM.
Re:With AI You Need to Think Much Bigger by inerte on Monday March 10, 2025 @12:55PM (#43322490)
I've been programming for 20 years, just so y'all know I have at least some level of competence.

I tried Vibe Coding for the first time on Friday and was blown away. It was awesome. I met some people (all programmers) at a party on Friday and excitedly told them about it. One of them tried, he loved.

Then yesterday I read a LinkedIn (Lunatic?) post about "Vibe Design", where PMs will just tell the computer what to do using some sort of visual language, where you create UI elements and drop them on a canvas, and AI makes your vision come true, etc, etc...

And my first thought was: "Wait a minute, I've seen this movie before. Back on late 90s / early 2000s it was the 4th generation programming language, and Visual Basic would allow anyone to code any system"...

And while it's true Visual Basic did allow a BUNCH of people to make money building and selling systems to video rental shops and hair saloons, programmers never went away.

I welcome anyone building more software. The tools will only get better. And programmers will adapt them, and it will make us better too, and we will still be needed.

Re:With AI You Need to Think Much Bigger by axkdev on Monday March 10, 2025 @01:29PM (#43322902)
> I'm excited to understand more about how aider creates a map of the repository and effectively compresses the information in ways similar to how I keep track of high level ideas.

Run /map. It's nothing fancy, really.

Re:With AI You Need to Think Much Bigger by xrd on Monday March 10, 2025 @02:15PM (#43323724)
Ok, I meant to say, I know about /map, but there is so much to explore here with an open source coding tool and LSP. It feels like you could build some very powerful ways to represent connections using a lot of cool tools that are in use inside our editors. More structured and more relevant that the poor representations I can keep inside my crappy head.