All tests pass, everything seemed to work. Would not have noticed of I didn't read through the code. Amount of other stupid shit I found is crazy.

To that end, my very first experience with OpenCode let to a `rm -rf /` within minutes, which I caught by chance watching the streaming output. Now I know what I'm doing, every agent is sandboxed, better tooling and models.

for me, if it's stupid, but it works, then it's a coincidence.

In essence, it is writing the code faster (great), but then also writing the code that was often previously never written due to budget/time limitations (comprehensive testing/validation).

So, how do you know the code is the highest quality ever? Because we have test result data that PROVES it to be the case. It isn't "trust but verify" it is "Don't trust. Verify."

It said verbatim: "Don't trust. Verify."

pgrust thought this too, but then another human discovered claude had hardcoded benchmark query plans.

Now pgrust has a "test" to make sure the Ai doesn't do this again (unless it decides this test is bad for the current goal)

https://github.com/malisper/pgrust/blob/38634b408dea373f3944...

Real verification seems beyond transformer capabilities.

Single data point, my Sentry notifications across more than a half dozen client projects when from 2-4 a month to basically 0. That's while shipping 20x _more_ code.

I used to work in Web agency for sizeable brands and not a single site I built lasted over a year because the companies changed concept regularly. The next agency would come in and tell them it was built wrong and couldn't be reused etc etc. We did the same, it was irrelevant how it was built and we didn't even look. This is the nature of most code. The problem is that very few people will ever be in a position to be writing verifiable important code, but every developer likes to cosplay that they are.

Most code is ultimately an, annoyance businesses need to meet a goal that will always be a cost. It's often worth treating it as such and writing things that are easy to spin up and teardown.

The people most vexed by AI dev appear to be the ones who think writing code is a pinnacle role, it's not even though over our lives the salaries would sometime suggest otherwise. That's ultimately going to be a short term blip in the landscape and most code will be one seen as the cost it is.

Yet I find myself using AI coding agents more and more. In this day and age, your choice is to either use them, or be stomped into the ground by programmers who do.

AI still can't write code at my level of quality, but it can do so at adequate quality now; and sadly, that is "good enough" for the real world.

FIFY.

I've found experienced devs with AI is a force multiplier, but AI in the hands of juniors however is my hovercraft is full of eels.

I'd say that 90% of them are very excited about using LLMs for coding. They have nothing to prove to anyone and can easily steer AI to reach "godly" productivity (please note these guys were 10x programmers already).

I also know some weak programmers and most of them tend to be critical of AI and want to get rid of it.

Lastly, a couple of people I know are against AI for moral/ideological reasons without denying it can bring productivity increases.

This is purely anecdotal but I'm starting to notice this pattern everywhere I look.

Sure, LLM's are unreliable. So are juniors.

Sure, LLM's code quality is not always great. Well, so is the juniors code.

The benefit of LLM's is that they can work super fast and have a vast amount of knowledge. The benefit of juniors is that they actually learn from their mistakes and get better over time. LLM's stay at intern/junior level, even with the correct context.

So for me, I see each LLM session as a new intern coming in. You hand them the right context, give them the correct instruction, and off they go. Then they come back, and you will have to judge how detailed you want to review what they produced. Or in other words, can you get away with some bad code? And for the next session, it's a new intern that starts from 0.

But it is also highly dependent on the project you are working on. If it's a big complex codebase, LLM's will struggle, and they can only do simple tasks (however, they can help with testing a lot!)

So the teachable thing is unfortunately experience:

- Know where quality matters, and where it doesn't

- Know what kind of mistakes could be made and should be checked for

- Know which things can be duck-taped together, because they live at the edge, and which things need a proper foundation, because the rest builts on it.

So in the end, I think it all comes down to making the right trade-offs and the right decisions (which LLM's are terrible at, by the way). And unfortunately that is gained by the slow process of experience.

I think it's very tough for junior developers right now, because they either go fast by letting the LLM's do the work, but then they learn nothing. Or they go slow by asking the LLM's what's going on, and then do learn.

A senior developer never understood the full code-base, and never reviewed every line of code. That's just impossible. And now every developer needs to be a senior developer that is able to steer these new "interns".

And just like with juniors and interns, you have to put a lot of effort into it, and it's not always clear in the end if you gained or lost something :D.

To comment on the post you linked:

I agree that people can be thought. But on the other hand, I also had juniors that reached their limit pretty fast, which can also be frustrating. The best way to "teach" LLM's is the context, but I agree it has limits and is not really teaching anything, just steering them in the right direction every time (like a checklist).

Your comment basically states that running LLM's is a gamble, but according to my experience, most of the time it delivers what I ask. Maybe in a different way that how I would do it, but I want results, and am not interested in micro-managing.

Are you working on a project that requires high quality code? Or is it your preference to work in such a codebase? For me, I was always ok with "good enough" code. And sometimes "good enough" needs to be good, and other times you can get away with some really bad code.

i meant "working with" as i explained in the linked comment. the feedback and response i get from juniors is very different. and yes, sure every junior dev is different from the others, but i simply enjoy working with people and i can't stand working with machines in the same way.

what bothers me about LLMs is that they are not deterministic. that just drives me nuts. i do enjoy learning the ins and outs of a language or compiler to figure out how to write the code to get the results i need. but LLMs just don't work like that. the same instruction that produced good working code today could produce nonsense tomorrow.

this is not about quality. this is about trust. i can't trust LLMs and that makes it very uncomfortable for me to work with them.

That's because you try to look at LLM's as tools, and not as an intern.

LLM's are not deterministic, they can make mistakes, etc. You refuse to work with LLM's like you would work with interns, and then somehow you're frustrated that they don't work like a compiler?

Maybe shifting your expectations would solve a lot of your frustrations.

Look, every developer writes bugs. Me, you, everyone. There is no perfect developer that can churn out 100% quality code. That means you always have to take this into account. You are correct that I don't take compiler bugs into account, because they are very rare. But when working with LLM's, just take into account that it can go wrong, just like any human developer.

Basically, you have very fast and cheap interns at your disposal, with a huge amount of knowledge. Can you trust an intern? Of course not!

There's nothing wrong with that. Many of us do (or did) like programming and the problem-solving itch it scratched for us. But getting work done for pay is more than what we like. If an LLM can reliably solve a problem for me in a day that would have taken a project, a team of several people, and close to a month of work to build out manually, then I go with the tool that works.

And that "if" is the reality I live in. About a decade ago, I wrote a program to do analysis of config files for a product our team uses. It took me weeks of side-work to implement the idea of passing boolean python functions as checks for the configuration. Such an elegant way of creating new checks on our system! It was company work, and when I left to a new place, I didn't have that code, just the idea. Fast forward five years and I want to write the same thing at a new place. I asked an LLM to do the same task, and it wrote exactly my solution immediately. It just worked. It shocked me. In the last few months, I've had it write analysis tools against this product that would have taken me days or weeks. I am admittedly not a comp-sci level programmer.

There is no denying that LLMs are powerful coding tools. That can be true at the same time that we lament the end of manual-coding-for-pay, the death of programming education as we know it, and the worries about employment and society.

No, but when a junior puts up a PR I do. They aren’t trustworthy enough yet.

One of the biggest parts of being senior is ownership of your code and knowing you’re putting your trustworthiness behind every PR you make. If you’re making bad PRs, you can’t say it’s Claude’s fault because you didn’t think a certain area was worth reviewing.

> That's just impossible.

Somewhat, but this is directionally bad advice. There’s huge payoffs for taking time to thoroughly understand areas you’re working in. It greatly amplifies your ability to prompt and overall execute in a codebase.

That's why I say: every session is a new intern. That's how I look at it.

> The claim is that if I'm any good, I should know how to squeeze consistent quality results from LLMs.

Claude Opus 5.5 can write decent code, better than most juniors I came across, and definitely codes faster than me.

But is that quality high enough for your standards? It sounds like no. So then the next question is, does your project really require your high standards?

Testing. It's all about having unit tests, and known outcomes that the LLM can check against every time it makes changes. For my work, that is good enough. For safety-critical systems I would demand more rigor in the testing and the human review and creation of tests.

I steer the implementation broadly by telling the model what libraries to use, what to pay attention to, where are the common source of bugs, gotchas, possible performance issues, security concerns, etc.

Let's say I want to add a good search functionality to my website, I wouldn't just say implement search, good way. I would say to use TypeSense, have that as a separate service, to benchmark performance on various queries, make use of caching, test cold starts, make sure autocomplete has good UI/UX on both desktop and mobile, to avoid layout shifting, display clear progress while search is loading, etc.

I don't care much how it implements them, as long as the result works as expected. Yes, there might be small bugs, but that happens when coding and testing manually too, and they will simply be fixed over time.

Competitive programming is rarely about writing code, it's mostly about knowing which data structure, libraries and ideas to use for each problem. When two competitive programmers talk, and talk about the solution, the solutions to even the hardest problems can often be described in just a few keywords. One would say "dynamic programming + binary search", and that's enough for the other guy to know exactly what the solution is and how to implement it. Same with LLMs, if you just give them a problem and ask them to solve it, they will likely do a shitty job, but once you just give one or two hints to put them on the right path, the solution is then almost perfect and as good as I would have implemented it, or even better, because when I implement it I might be lazy and cut corners to write code faster, but the LLM can organize and test code at such speed that it comes at basically no cost.

The first months of using AI were tough, because suddenly I felt like I lost control over my creations, and something else was changing them in unkown ways, but now I learned how to control data and use that to my advantage.

Is like you were manually planting flowers, each one carefully placed, but now you got a tractor that plants 10000 flowers per hour, in a slightly worse way, but then you learn how to set-up the tractor so the spacing is right, that they are aligned and planted at the right depth, etc. Then you realize this opens a whole new world, now you can not only plant a few plants, but make fields of flowers, beautiful art that would have taken an impossible amount of time before is now possible.

The only danger is losing grasp of internal components of the system, but maybe there'll be a "management" layer eventually for humans.

I don't know what the future of programming will look like but I don't see us rolling back the clock to 2021..

A good human code review takes into account the broader business context and the structure of the codebase. It finds instances of poor design or readability. It suggests patterns or existing utilities that the author didn’t make use of, and points out antipatterns.

LLM reviews are a great first line of defense against certain classes of bugs. But I consistently find major issues in code review that the review bots miss — and some that they couldn’t have caught under any circumstances.

Just last week: 800 lines of unnecessary (and clearly LLM-written) code with high cyclomatic complexity that could be easily replaced by 50 lines of mostly declarative code by using existing patterns and utilities; several missed opportunities for converting incoming data to native types up front upon receipt and enforce stricter typing, rather than passing around things like dates or UUIDs (or even basic numeric types) as strings and trying to parse them over and over; and a case where the author had misunderstood which of two possible sources certain data should be derived from, resulting in subtly incorrect data being saved to the system of record.

A phrase I've been toying with is eyes free vs hands free coding.

The distinction is important. While I too hardly write code these days, I do continue to make smaller changes and look at what's being done. This is as opposed to the group who has decided vibing mega commits is a good idea.

High quality code doesn't always lead to a high quality product.

And bad code doesn't always lead to a bad product.

And performance is easy to test automatically by having performance benchmarks automatically ran on deployment.

It seems most people are bad at evaluating the quality of their work, especially if they purchased it from Anthropic.

It could have also been a "use AI to test our product more", but I don't see that anywhere.

I hope you agree LLM's can actually help with quality, if used for that purpose?

And it's not just restricted to software. Everything I care about is getting worse.

Dribbling about how it could theoretically be better doesn't help undo the damage.

I do think/hope that as LLMs improve, that improved quality would come by default, or that people would still compete in quality instead of quantity, but this is also the consumer's job to not accept shit quality products.

It doesn't happen. We have very good functional and non-functional testing and the coding agents are able to iteratively add functionality correct features and improve performance.

I don't understand what other people are doing different to get worse results. We cannot ever go back to humans writing this code, because our competitors will eat our lunch if we do.

Well that doesn't make sense if you are writing code that is going to be used on critical infrastructure.

Imagine a disaster were to happen and you said to the auditors that "I don't know what I am doing nor did I look at the code" and you cost the company tens of millions of dollars.

Sure if your building toys, that's fine, otherwise, why take the risk in being liable for not looking at what the code does and then believing it is high quality?

Don't you see the contraction you just did there?

Wonder if he tells his boss to "I pushed critical code to production without even taking a look at it" or is it just for show on HN

If there is a bug, it will be fixed quickly anyway, so only a few users might ever see it, or see it only for a brief time.

Even when manually coding, projects still have issues and bugs, I don't think much changes in that aspect now, if anything, there are fewer bugs now in most high quality apps than before, due to AI catching more issues. See all the vulnerabilities found in old core code and libraries nowadays, that humans didn't find.

>>Sure if your building toys, that's fine, otherwise, why take the risk in being liable for not looking at what the code does and then believing it is high quality?

>>Don't you see the contraction you just did there?

I wouldn't trust anyone who refuse to use LLMs for programming. Their code is worse, they test it less and it takes longer to write. For me "no LLMs" attitude is a sign of incompetence. Good for hobbyist trying to keep their brain active but that's about it.

That is even more so for critical infrastructure. Just look at the mess human coding is when it comes to security. LLMs are fixing it at the pace we couldn't even imagine a year or so ago.

So does that mean you should never look at its code?

If a fault in the software occurs even after testing and there was an investigation, will you admit in the postmortem report that you did not look at the code at all and blame the agent for the error?

> I wouldn't trust anyone who refuse to use LLMs for programming.

Did I say I refuse to use LLMs for programming? I am telling you to check the agent's work.

I would not trust anyone who DOES NOT check the changes of an agent's code when using them for programming in highly regulated industries or in critical systems. For most companies in the Fortune 500, it is irresponsible to not check.

> That is even more so for critical infrastructure. Just look at the mess human coding is when it comes to security. LLMs are fixing it at the pace we couldn't even imagine a year or so ago.

No-one argued against that and it does not mean that you do not check with the agent at all even if the agent believes that they "fixed it".

We have more freedom than ever to build whatever we dream of, you still need a dreamer. And that’s why the profession of coding isn’t going away, you still need people to turn ideas into reality. I find AI soulless, it’s great at mimicking but is incapable of coming up with an original idea that wasn’t yours regurgitated.

This thread is full of 100x supercharged programmers, so why is it that I can't name a single useful feature added in the last 5 years in e.g. macOS, Windows, Firefox, Chrome, YouTube, VS Code, YouTube, Slack, Photoshop, Word or any of the other 5 programs currently on my taskbar?

What I have seen is that, with LLMs, an entire class of process brokenness disappeared in the in-house codebase. Before, there were many trivial tickets that, in the end, required a 5-line bugfix sitting in the queue for months. Now, such tickets are resolved instantly, and the backlog is shrinking.

Personally, I did see a lot of entry-level products having a lot higher quality and better UX. At the same time, most companies go for fast iterations and are profit-focused, do product quality is not really a target for many.

None of this capital has ever been earmarked to build better consumer utilities.