News for hackers. Stuff that matters.
Re:As an experienced LLM user, I don't use generative LLMs o... by surgical_fire on Monday May 05, 2025 @03:13PM (#43898402)
This has been my experience with any LLM I use as a code assistant. Currently I mostly use Claude 3.5, although I sometimes use Deepseek or Gemini.

The more prominent and widely used a language/library/framework, and the more "common" what you are attempting, the more accurate LLMs tends to be. The more you deviate from mainstream paths, the more you will hit such problems.

Which is why I find them them most useful to help me build things when I am very familiar with the subject matter, because at that point I can quickly spot misconceptions, errors, bugs, etc.

It's when it hits the sweet spot of being a productivity tool, really improving the speed with which I write code (and sometimes improving the quality of what I write, for sometimes incorporating good practices I was unaware of).

Re:As an experienced LLM user, I don't use generative LLMs o... by steveklabnik on Monday May 05, 2025 @03:21PM (#43898483)
> The more prominent and widely used a language/library/framework, and the more "common" what you are attempting, the more accurate LLMs tends to be. The more you deviate from mainstream paths, the more you will hit such problems.

One very interesting variant of this: I've been experimenting with LLMs in a react-router based project. There's an interesting development history where there's another project called Remix, and later versions of react-router effectively ate it, that is, in December of last year, react-router 7 is effectively also Remix v3 https://remix.run/blog/merging-remix-and-react-router

Sometimes, the LLM will be like "oh, I didn't realize you were using remix" and start importing from it, when I in fact want the same imports, but from react-router.

All of this happened so recently, it doesn't surprise me that it's a bit wonky at this, but it's also kind of amusing.

Re:As an experienced LLM user, I don't use generative LLMs o... by joshuacc on Monday May 05, 2025 @08:35PM (#43900777)
I ran into this as well, but now I have given standing instructions for the llm to pull the latest RR docs anytime it needs to work with RR. That has solved the entire issue.
Re:As an experienced LLM user, I don't use generative LLMs o... by zoogeny on Monday May 05, 2025 @03:50PM (#43898763)
In addition to choosing languages, patterns and frameworks that the LLM is likely to be well trained in, I also just ask it how it wants to do things.

For example, I don't like ORMs. There are reasons which aren't super important but I tend to prefer SQL directly or a simple query builder pattern. But I did a chain of messages with LLMs asking which would be better for LLM based development. The LLM made a compelling case as to why an ORM with a schema that generated a typed client would be better if I expected LLM coding agents to write a significant amount of the business logic that accessed the DB.

My dislike of ORMs is something I hold lightly. If I was writing 100% of the code myself then I would have breezed past that decision. But with the agentic code assistants as my partners, I can make decisions that make their job easier from their point of view.