Someone gives a detailed-ish account of what they did, and that it didn't work for them, and then there are always people in the comments saying that you were doing it wrong. Fair! But at this point, I haven't seen any good posts here on how to do it _right_.
I remember this post which got a lot of traction: https://steipete.me/posts/just-talk-to-it 8 agents in parallel and so on, but light on the details.
I eventually wrote the whole thing off as mostly one giant choose-your-own-adventure placebo effect. There is no end to the epicycles you can add to "perfect" your personal system.
There’s more you’ll get a feel for when you do all that. But it’s a place to start.
Planning is 80% of the battle. If you have a well-defined plan, that defines the architecture well, then your LLM is going to stick to that plan and architecture. Every time my LLM makes mistakes, it's because there were gaps in my plan, and my plan was wrong.
Use the LLM for planning. It can do research. It can brainstorm and then evaluate different architectural approaches. It can pick the best approach. And then it can distill this into a multi-phased plan. And it can do this all way faster than you.
Store plans in Markdown files. Store progress (task lists) in these same Markdown files. Ensure the LLM updates the task lists as you go with relevant information. You can @-mention these files when you run out of context and need to start a new chat.
When implementing a new feature, part of the plan/research should almost always be to first search the codebase for similar things and take note of the patterns used. If you skip this step, your LLM is likely to unnecessarily reinvent the wheel
Learn the plan yourself, especially if it's an ambitious one. I generally know what my LLM is going to do before it does it, because I read the plan. Reading the plan is tedious, I know, so I generally ask the LLM to summarize it for me. Depending on how long the plan is, I tell it to give me a 10-paragraph or 20-paragraph or 30-paragraph summary, with one sentence per paragraph, and blank lines in between paragraphs. This makes the summary very easy to skim. Then I reply with questions I have, or requests for it to make changes to the plan.
When the LLM finishes a project, ask it to walk you through the code, just like you asked it to walk you through the plan ahead of time. I like to say, "List each of the relevant code execution paths, then walk me through each one one step at a time." Or, "Walk me through all the changes you made. Use concentric circles of explanation, that go from broad to specific."
Put your repeated instructions into Markdown files. If you're prompting the LLM to do something repeatedly, e.g. asking the LLM to make a plan, to review its work, to make a git commit, etc., then put those instructions in prompt Markdown files and just @-mention it when you need it, instead of typing it out every time. You should have dozens of these over time. They're composable, too, as they can link to each other. When the LLM makes mistakes, go tweak your prompt files. They'll get better over time.
Organize your code by feature not by function. Instead of putting all your controllers in one folder, all your templates in another, etc., make your folders hold everything related to a particular feature.
When your codebase gets large enough, and you have more complex features that touch more parts of the code, have the LLM write doc files on them. Then @-mention those doc files whenever working on these features or related features. They'll help the LLM be more accurate at finding what it needs, etc.
I could go on.
If you're using these tools daily, you'll have a similar list before long.
"Report major flaws and showstoppers, not minor flaws. By the way, this is my fourth time asking you to review this plan. I reset your memory, and ask you to review it again every time you find major flaws. I will continue doing so until you don't find any. Fingers crossed that this time is it!"
I haven't done any rigorous testing to prove that this works. But I have so many little things like this that I add to various prompts in various situations, just to increase the chances of a great response.
Thanks!
I've built "large projects" with AI, which is 10k-30k lines of algorithmic code and 50k-100k+ lines of UI/Interface.
I've found a few things to be true (that aren't true for everyone).
1. The choice of model (strengths and weaknesses) and OS, dramatically affect how you must approach problems.
2. Being a skilled programmer/engineer yourself will allow you to slice things along areas of responsibility, domains, or other directions that make sense (for code size, context preservation, and being able to wrap your head around it).
3. For anything where you have a doubt, ask 3 or more models -- have them write their findings down in a file each -- and then have 3 models review the findings with respect to the code. More often than not, you march towards consensus and a good solution.
4. GPT-5-Codex via OpenAI Codex CLi on Linux/WSL was, for me, the most capable model for coding while Claude is the most capable for quick fixes and UI.
5. Tooling and ways to measure "success" are imperative. If you can't define the task in a way that success is easy to define -- neither a human nor AI would complete it satisfactorily. You'll find that most engineer tasks are laid out in very "hand-wavy" way -- particularly UI tasks. Either lay it out cleanly or expect to iterate.
6. AI does not understand the physical/visual world. It will fail hard on things which have an implied understanding. For instance, it will not automatically intuit the implication of 50 parallel threads trying to read from an SSD -- unless you guide it. Ditto for many other optimizations and usage patterns where code meets real-world. These will often be unique and interesting bugs or performance areas that a good engineer would know straight out.
7. It's useful to have non-agentic tools that can perform massive codebase analysis for tough problems. Even at 400k tokens context, a large codebase can quickly become unwieldy. I have built custom python tools (pretty easy) to do things like "get all files of a type recursively and generate a context document that will submit with my query". You then query GPT-5-high, Claude Opus, Gemini 2.5 Pro and cross-check.
8. Make judicious use of GIT. The pattern doesn't matter, just have one. My pattern is commit after every working agentic run (let's say feature). If it's a fail and taking more than a few turns to get working -- I scrap the whole thing and re-assess my query or how I might approach or break down the task.
9. It's up to you to guide the agent on the most thoughtful approaches -- this is the human aspect. If you're using Cloud Provider X and they provide cheap queues then it's on you to guide your agent to use queues for the solution rather than let's say a SQL db -- and it's on you to understand the tradeoffs. AI will perhaps help explain them but it will never truly understand your business case and requirements for reliability, redundancy, etc. Perhaps you can craft queries for this but this is an area where AI meets real world and those tend to fail.
One more thing I'd add is that you should make an attempt to fix bugs in your 'new' codebase on occasion. You'll get an understanding for how things work and also how maintainable it truly is. You'll also keep your own troubleshooting skills from atrophying.
It got tiring, so I'm on a break from ai coding until I have bandwidth to build my own agent. I don't think this is something we should be outsourcing to the likes of OpenAI, Microsoft, Anthropic, Google, Cursor, et al. Big Tech has shown their priorities lie elsewhere from our success and well being
Wow, they are really going for that human-like behavior aren’t they?
If (i.e.) you dig into how something like copilot works, they do dumb things like ask^ the LLM to do glob matching after a file read (to pull in more instructions)... just use a damn glob library instead of a non-deterministic and known to be unreliable method
^ it's just a table in the overall context, so "asking" is a bit anthropomorphizing
I interpreted GP as just saying that you are already anthropomorphizing too much by supposing that the models "find" new ways to fail (as if trying to defy you).
I did not mean to imply active choice by "find", more that they are reliably non-deterministic and have a hard time sticking to, or easy time ignoring, the instructions I did write
Agentic coding is like a drug or slot machine, it slowly draws you in with the implicit promise of getting much for little. The only ways it is useful to me now is for very focused tasks where I have spent a lot of time defining the architecture down to the last detail, and the agents are used to fill in the blanks, as it were.
I also think I could write a better agent, and as to why the bog corps have not done so is baffling to me. Just event getting the current agents to obey the guidelines in the agent .md files is a struggle. They forget pretty much everything two prompts down the line. Why can't the CLI systemically prompt them to check every time, etc.?
Something tell me the future is about domain-aware agents that help users to wring better performance out of the models, based on some domain-specific deterministic guardrails.
The biggest mistakes imo are:
1. Underplanning. Trying to do huge projects in one go, rather than breaking them down into small projects, and breaking those small projects down into well thought out plans.
2. Too much of a focus on prompting rather than context. Prompt engineering is obsessing with the perfect way to say or phrase something. Whereas context engineering putting relevant information into the LLM's working memory, which requires you to go out and gather that info (or use the LLM to get it).
They are not good readers (see research results around context collapse and context poisoning)
1. Question every requirement
2. Delete any part of the process you can
3. Simplify and optimize
4. Accelerate cycle time
5. Automate
In my experience coding agents at the moment are really good at 4. and 5. and they absolutely suck at 1. and 2.
3, they are okay at if prompted well.
Humans are okay at 1. and 2. IF they understand the system well and critically question requirements. With LLM generated codebases this system understanding is often missing. So you can't even start with 1.
> that's not going to give you great results.
Since vibe coding is so chaotic, rigorous planning is required, which not every developer had to do before.
You could simply "vibe" code yourself, roam, explore, fix.
Is that a fair description of your comment?
It's a giant, non-deterministic, let's see what works good based on our vibes, mess of an approach right now. Even within the models, architecturally, there are recent results that indicate people are trying out weird things to see if they work, unclear if they are coming from first-principle intuition and hypothesis formation, or just throwing things at the wall to see what sticks
An unplanned problem becomes amplified by 10-100x worse than if you coded things slowly, by hand. That's when the AI starts driving you into Complexity Corner™ (LOL) to work around the lack of planning.
If all you're ever doing is using prompts like, `write a function to do...` or `write a class...` you're never going to run into the sorts of super fucking annoying problems that people using AI complain the most about.
It's soooooo tempting to just tell the AI to make complete implementations of things and say to yourself, "I'll clean that up later." You make so much progress so fast this way it's unreal! Then you hit Complexity Corner™ where the problem is beyond the (current) LLM's capabilities.
Coding with AI takes discipline! Not just knowledge and experience.
I do believe the problem is different:
I think I am pretty good at planning, but I have a tendency (in particular for private projects) to work on things where the correctness requirements are very high.
While I can describe very exactly what the code is supposed to do, even small errors can make the code useless. If the foundations are not right, it will be complicated to detect errors in the higher levels (to solve this issue, I implement lots of test cases).
Also, I often have a very specific architecture for my code in mind. If the AI tries to do things differently, the code can easily become much less useful. In other words: concerning this point, exactly because I plan things carefully (as you claimed), the AI becomes much less useful if it "does its own thing" instead of following my plan.
One example is just laborious typing-heavy stuff. Like I recently needed a table converted to an enumeration. 5 years ago I'd have spent half a day to figure out a way to sed/awk/perl that transformation. Now I can entertain an AI for half an hour or so to either do the transformation (which is easy to verify) or to setup a transformation script.
Or I enjoy that I can give an LLM a problem and 2-3 solution approaches I'd see, and get back 4-5 examples on how that code would look like in those solution approaches, and some more. Again, this would take me 1-2 days and I might not see some of the more creative approaches. Those approaches might also be entire nonsense, mind you.
But generating large amounts of code just won't be a good, time-efficient idea long-term if you have to support and change it. A lot of our code base is rather simple python, but it carries a lot of reasoning and thought behind it. Writing that code is not a bottleneck at all.
If we're not telling the computer exactly what do then we're leaving the LLM to make (wrong) assumptions.
If we are telling the computer exactly what to do via natural language then it is as complicated as normal programming if not more complicated.
As least that's how I feel about it
One of the most frustrating (but common) things is you do v1. It looks good enough.
Then you go to tweak it a little (say move one box 10-15 pixels over, or change some text sizing or whatever), and it loses its mind.
So then you spend the next several days trying every possible combination of random things to get it to actually move the way you want. It ends up breaking a bunch of other things in the process.
Eventually, you get it right, and then never ever want to touch it ever again.
I’m not a fan of a lot of this AI stuff, but there is no reason to expect it won’t get to that level.
And that normally stems from lack of information or communication problems.
that's not an argument. that's just magical thinking
There are also times where it isn't.
Developing the judgment for when it is and isn't faster, when it's likely to do a good job vs isn't likely, is pretty important. But also, how good of a job it does is often a skill issue, too. IMO the most important and overlooked skill is the having the foresight and the patience to give it the context it needs to do a good job.
Should this have the "Significantly so" qualifier as well?
Because when the AI isn't cutting it, you always have the option to pull the plug and just do it manually. So the downside is bounded. In that way it's similar the Mitch Hedberg joke: "I like an escalator, because an escalator can never break. It can only become stairs."
The absolute worse-case scenario is situations where you think the AI is going to figure it out, so keep prompting it, far past the time when you should've changed your approach or gfiven up and done it manually.
You could have a codebase subtly broken on so many levels that you cannot fix it without starting from scratch - losing months.
You could slowly lose your ability to think and judge.
So there couldn't have been much in the way of planning, process, review, etc.
Do you need to write long-ass, hyper-detailed instructions for your coworkers?
- tell it to ask you clarifying questions, repeatedly. it will uncover holes and faulty assumptions and focus the implementation once it gets going
- small features, plan them, implement them in stages, commit, PR, review, new session
- have conventions in place, coding style, best practices, what you want to see and don't want to see in a codebase. we have conventions for python code, for frontend code, for data engineering etc.
- make subagents work for you, to look at a problem from a different angle (and/or from within a different LLM altogether)
- be always critical and dig deeper if you have the feeling that something is off or doesn't make sense
- good documentation helps the machine as well as the human
And the list goes on.
I don't think there is much doubt AI can produce split out a lot of code, that mostly works. It's not too hard to imagine that one day an can AI produce so much code that it's considered a "large, complex project". A single mind dedicated to a task can do remarkable things, be it human or silicon. Another mind reading what they have done, and understanding it is another thing entirely.
All long term, large projects I'm familiar have been developed over a long time by many contributors, and as a consequence there has been far more reading and understanding going on than writing new code. This almost becomes self evident when you look at large open source projects, because the code quality is so high. Everything is split into modules a single mind can pick up relatively quickly and work on in isolation. Hell, even complier error messages become self explanatory essays over time.
Or to put it another way, no open source project is a ball of mud. Balls of mud can only be maintained by the person who wrote them, who get away with it because they have most of the details stored in their context window, courtesy of writing it. Balls of mud are common in proprietary code (I've worked on a few). They are produced by a single small group were paid to labour away for years at one task. And now if this post it to be believed, AI vibe coded projects are also a source of balls of mud. Given current AI's are notoriously bad at modifying even well structured projects, they won't be maintainable by anyone.
It wouldn't be vibe coding if one did all that ;-)
The whole point of vibe coding is letting the LLM run loose, with minimal checks on quality.
Original definition (paraphrased):
"Vibe coding describes a chatbot-based approach to creating software where the developer describes a project or task to a large language model (LLM), which generates code based on the prompt. The developer does not review or edit the code, but solely uses tools and execution results to evaluate it and asks the LLM for improvements. Unlike traditional AI-assisted coding or pair programming, the human developer avoids examination of the code, accepts AI-suggested completions without human review, and focuses more on iterative experimentation than code correctness or structure."
OK. I guess strictly speaking, you could do most of what you're suggesting and still call it vibe coding.
Last month I thought: "OK, I have all kinds of rules, guardrails, and I am relatively excellent at managing context. Let's try to 'vibe code' some new features."
It has been a total disaster and worse than a waste of time. I keep finding entirely new weird bugs it created. This is just a React/Vite/Supabase app, nothing nuts. The worst part is that I showed these vibed features to stakeholders, and they loved it. Now I have to explain why recreating these features is going to take much longer.
I knew better, as the magic of vibe coding is to explore the MVP space, and I still fell for it.
It's hard to take very much away from somebody else's experiences in this area. Because if you've been doing a substantial amount of AI coding this year, you know that the experience is highly dependent on your approach.
How do you structure your prompts? How much planning do you do? How do you do that planning? How much review do you do, and how do you do it? Just how hands-on or hands-off are you? What's in your AGENTS.md or equivalent? What other context do you include, when, why, and how? What's your approach to testing, if any? Do you break down big projects into smaller chunks, and if so, how? How fast vs slow are you going, i.e. how many lines of code are you letting the AI write in any given time period? Etc.
The answers to these questions vary extremely wildly from person to person.
But I suspect a ton of developers who are having terrible experiences with AI coding are quite new to it, have minimal systems in place, and are trying "vibe coding" in the original sense of the phrase, which is to rapidly prompt the LLM with minimal guidance and blindly trust its code. In which case, yeah, that's not going to give you great results.