News for hackers. Stuff that matters.
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by janalsncm on Friday April 24, 2026 @04:50PM (#47895612)
How can a person reconcile this comment with the one at the root of this thread? One person says Claude struggles to even meet the strict requirements of a spec sheet, another says Claude is doing a great job and doesn’t even need specific specs?

I have my own anecdata but my comment is more about the dissonance here.

Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by sarchertech on Friday April 24, 2026 @05:33PM (#47896079)
One person is rigorously checking to see if Claude is actually following the spec and one person isn’t?
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by hunterpayne on Friday April 24, 2026 @06:18PM (#47896504)
One is getting paid by a marketing department program and the other isn't. Remember how much has been spent making LLMs and they have now decided that coding is its money maker. I expect any negative comment on LLM coding to be replied to by at least 2 different puppets or bots.
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by riquito on Friday April 24, 2026 @06:33PM (#47896637)
Then you should expect any positive comment to be replied negatively by a competition's puppet or bot too
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by SpaceNoodled on Saturday April 25, 2026 @10:57AM (#47901982)
Not necessarily; rising tide and all that. When a new scam like this emerges, it behooves all of the grifters to cooperate and not muddy the waters with distrust.
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by sarchertech on Saturday April 25, 2026 @11:54AM (#47902374)
I’m normally very skeptical of conspiracy theories. But saw an AI booster bot responding to a negative AI post I made here.

Someone pointed out to me in the comments that the username had posted long replies to 3 completely different threads in the same minute. That and looking back at its post history confirmed it was a bot.

Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by flyinglizard on Friday April 24, 2026 @06:06PM (#47896390)
... or one person has a very strong mental model of what he expects to do, but the LLM has other ideas. FWIW I'm very happy with CC and Opus, but I don't treat it as a subordinate but as a peer; I leave it enough room to express what it thinks is best and guide later as needed. This may not work for all cases.
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by sarchertech on Friday April 24, 2026 @06:40PM (#47896701)
If you don’t have a very strong mental model for what you are working on Claude can very easily guide in you into building the wrong thing.

For example I’m working on a huge data migration right now. The data has to be migrated correctly. If there are any issues I want to fail fast and loud.

Claude hates that philosophy. No matter how many different ways I add my reasons and instructions to stop it to the context, it will constantly push me towards removing crashes and replacing them with “graceful error handling”.

If I didn’t have a strong idea about what I wanted, I would have let it talk me into building the wrong thing.

Claude has no taste and its opinions are mostly those of the most prolific bloggers. Treating Claude like a peer is a terrible idea unless you are very inexperienced. And even then I don’t know if that’s a good idea.

Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by oops on Friday April 24, 2026 @07:53PM (#47897231)
That’s interesting to hear as for me Claude has been quite good about writing code that fails fast and loud and has specifically called it out more than once. It has also called out code that does not fail early in reviews.
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by sarchertech on Saturday April 25, 2026 @12:00PM (#47902424)
If you add a single space to a prompt, you’ll get a completely different output, so it’s no surprise that feeding entirely different programs into the prompt produces radically different output.

My guess is that there must be something about the language(go) or the domain (a data migration tool that uses Kafka) that triggers this.

Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by aforwardslash on Friday April 24, 2026 @09:01PM (#47897667)
Have you created a plan where the requisite is not to bother you with x and y, and to use some predetermined approach? What you describe sometimes happens to me, but it happens less when its part of the spec.
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by sarchertech on Saturday April 25, 2026 @12:02PM (#47902442)
Yes. That’s one of the things included in this.

> No matter how many different ways I add my reasons and instructions to stop it to the context

Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by timr on Friday April 24, 2026 @09:28PM (#47897805)
> Claude has no taste and its opinions are mostly those of the most prolific bloggers.

I often think that LLMs are like a reddit that can talk. The more I use them, the more I find this impression to be true - they have encyclopedic knowledge at a superficial level, the approximate judgement and maturity of a teenager, and the short-term memory of a parakeet. If I ask for something, I get the statistical average opinion of a bunch of goons, unconstrained by context or common sense or taste.

That’s amazing and incredible, and probably more knowledgeable than the median person, but would you outsource your thinking to reddit? If not, then why would you do it with an LLM?

Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by prmph on Saturday April 25, 2026 @05:35AM (#47900051)
> they have encyclopedic knowledge at a superficial level, the approximate judgement and maturity of a teenager, and the short-term memory of a parakeet. If I ask for something, I get the statistical average opinion of a bunch of goons, unconstrained by context or common sense or taste.

Love this paragraph; it's exactly how I feel about the LLMs. Unless you really know what you are doing, they will produce very sub-optimal code, architecturally speaking. I feel like a strong acumen for proper software architecture is one of the main things that defines the most competent engineers, along with naming things properly. LLMs are a long, long way from having architectural taste

Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by flyinglizard on Saturday April 25, 2026 @06:33AM (#47900325)
Try asking to review your code as if it were Linus Torvalds. No, really.
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by sarchertech on Saturday April 25, 2026 @11:56AM (#47902405)
I’ve tried that. I’ve experimented with a whole council of 13 personas including many famous developers. It’s definitely different. But it’s hasn’t performed significantly better in my tests.
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by datavirtue on Saturday April 25, 2026 @07:44AM (#47900673)
Holding it wrong.
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by justinclift on Saturday April 25, 2026 @01:36AM (#47898954)
> it will constantly push me towards removing crashes and replacing them with “graceful error handling”.

Is it generating JS code for that?

Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by sarchertech on Saturday April 25, 2026 @12:01PM (#47902430)
No this is a kafka consumer written in go.
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by flyinglizard on Saturday April 25, 2026 @06:32AM (#47900312)
You're right, data migration is a specific case where you have a very strong set of constraints.

I, on the other hand, am doing a new UI for an existing system, which is exactly where you want more freedom and experimentation. It's great for that!

Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by aforwardslash on Friday April 24, 2026 @08:58PM (#47897649)
It boils down to scope. I use CC in both very specific one-language systems and broad backend-frontend-db-cache systems. You can guess where the difficulty lies. (Hint: its the stuff with at least 3 distinct languages)
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by oefrha on Friday April 24, 2026 @09:35PM (#47897832)
One aspect you have to consider is the differences in human beings doing the evaluation. I had a coworker/report who would hand me obvious garbage tier code with glaring issues even in its output, and it would take multiple iterations to address very specific review comments (once, in frustration, I showed a snippet of their output to my nontechnical mom and even my mom wtf’ed and pointed out the problem unprompted); I’m sure all the AI-generated code I painstakingly spec, review and fix is totally amazing to them and need very little human input. Not saying it must be the case here, that was extreme, but it’s a very likely factor.
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by tclancy on Friday April 24, 2026 @10:56PM (#47898209)
This is magical because you are both on the exact right path and not right. My theory is there’s a sort of skill to teasing code from AI (or maybe not and it’s alchemy all over again) and this is all new enough and we don’t have a common vocabulary for it that it’s hard for one person who is having a good experience and one person who is not to meaningfully sort out what they are doing differently.

Alternatively, it could be there’s a large swath of people out there so stupid they are proud of code your mom can somehow review and suggest improvements in despite being nontechnical.

Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by onlyrealcuzzo on Monday April 27, 2026 @02:42PM (#47925516)
> This is magical because you are both on the exact right path and not right. My theory is there’s a sort of skill to teasing code from AI (or maybe not and it’s alchemy all over again) and this is all new enough and we don’t have a common vocabulary for it that it’s hard for one person who is having a good experience and one person who is not to meaningfully sort out what they are doing differently.

I don't think this is a hypothesis.

Outside of asking for one-shot tasks that have been done a million times before, LLMs do not "default" to good work.

If you ask them over-and-over again to find holes in their solution, to fix them, to evaluate for tech debt, to test all cases, to re-asses after the cases if it's architecturally coherent, to compare to the closest available known good implementations, etc etc, they can eventually get what you want done unbelievably cheaply to an acceptable level of quality.

I mentioned initially - their work is unbelievably cheap, you should be EAGER to reject it. Most people wouldn't even bend down to pick a penny up off the sidewalk. They can literally pump out CLs for a penny. You shouldn't even waste time looking at "I'm done" until they've gone through 10+ rounds of reviews, refactors, bug fixes, thought of more test cases, compared to known implementations, etc.

Why are you going to spend ~$50-$100+ of your time reviewing $0.01 of LLM time?! It makes no sense!

If you just listen to them say "I'm done" and move on to their next task, it won't take too many days before you're swimming in a sea of incoherent garbage.

Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by rhubarbtree on Saturday April 25, 2026 @04:50AM (#47899831)
This is plausible. Assuming it’s true, we would see the adoption of vibe coding at a faster rate amongst inexperienced developers. I think that’s true.

A counterpoint is Google saying the vast majority of their code is written by AI. The developers at Google are not inexperienced. They build complex critical systems.

But it still feels odd to me, this contradiction. Yes there’s some skill to using AI but that doesn’t feel enough to explain the gap in perception. Your point would really explain it wonderfully well, but it’s contradicted by pronouncements by major companies.

One thing I would add is that code quality is absolutely tanking. PG mentioned YC companies adopted AI generated code at Google levels years ago. Yesterday I was using the software of one such company and it has “Claude code” levels of bugginess. I see it in a bunch of startups. One of the tells is they seem to experience regressions, which is bizarre. I guess that indicates bugs with their AI generated tests.

Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by SpaceNoodled on Saturday April 25, 2026 @10:55AM (#47901969)
You don't think Sundar would do that, just go on the Internet and tell lies?
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by Degorath on Wednesday April 29, 2026 @02:50PM (#47952674)
Fairly certain they do something like Anthropic does, they count the acceptance rate or something else that is fairly "optimistic" (my org has a code acceptance rate of 98,5% per the platform dashboard). So, to clarify, me accepting the suggestion and then correcting it by hand still counts as N LoC accepted.
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by DennisP on Friday April 24, 2026 @10:19PM (#47898047)
I just read Steve Yegge's book Vibe Coding, and he says learning to use AI effectively is a skill of its own, and takes about a year of solid work to get good at it. It will sometimes do a good job and other times make a mess, and he has a lot of tips on how to get good results, but also says a lot of it is just experience and getting a good feel for when it's about to go haywire.
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by sameerds on Friday April 24, 2026 @11:33PM (#47898390)
It could just be that each of the two reviewers is merely focussing on different sides of the same coin? I use Claude all the time. It saves me a lot of effort that I would have otherwise spent in looking up specific components. The magically autocompleted pieces of boilerplate are a tangible relief. It also catches issues that I missed. But when it is wrong, it can be subtly or embarassingly or spectacularly wrong depending on the situation.
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by justinclift on Saturday April 25, 2026 @01:37AM (#47898964)
Note that one person is mentioning they use Claude Sonnet, which is less capable than the higher tiers (Opus, etc).
Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by mojuba on Saturday April 25, 2026 @03:00AM (#47899354)
I think it depends on both the complexity and the quality bars set by the engineer.

From my observations, generally AI-generated code is average quality.

Even with average quality it can save you a lot of time on some narrowly specialized tasks that would otherwise take you a lot of research and understanding. For example, you can code some deep DSP thingie (say audio) without understanding much what it does and how.

For simpler things like backend or frontend code that doesn't require any special knowledge other than basic backend or frontend - this is where the bars of quality come into play. Some people will be more than happy with AI generated code, others won't be, depending on their experience, also requirements (speed of shipping vs. quality, which almost always resolves to speed) etc.

Re:I Cancelled Claude: Token Issues, Declining Quality, and ... by stevenicr on Sunday April 26, 2026 @05:07AM (#47908706)
I think it matters what the project and tech stack is, and how much you try to get done before starting a fresh chat.

I've had interesting chats where it explained that it's choice of tailwind for example was because it had a ton of training knowledge on it.

I've also had it try to build more in one chat than it should many times.

For some reason openai codex handles building too much without failing better - but that is total anecdata from my particular projects and ymmv.

I've had these things try to build big when a little nudge gets them to change direction and not build so much. Explain which libraries and such and asking it to change the tech stack and the steps to build at once seem to make things much better for my use cases.

Also running extra checks and cleanup later is a thing, that sure a human might have seen an obvious thing at time of build, but we have bigger memory context comparatively imho.