boorang

I did the same in about the same timeframe. I created a nice little hierarchy of "when writing tests, read docs/test-patterns.md" type logic. The idea was to put all of my personal heuristics about coding in a "lazy loaded" documentation directory. I'm able to generate Java code that is nearly the same as what I would generate if I was coding it myself. I haven't deleted my documentation or anything. I see people making this claim, did you literally delete your coding documentation or something?

It seems like, for example, some people like to use var and some people like to use the full type name in Java declarations. The LLM can't know your preference upfront, you need to tell it. When people talk about shitty vibe-code I assume it's because they didn't do these steps.

I could see (but wouldn't count on it) the argument that the current models are so good at recognizing patterns in the code that in a mature well-organized codebase they don't need to be told what the patterns are, but I haven't bothered with those experiments yet. If it ain't broke don't fix it was my thinking.

I didn't think it was that controversial actually. I don't bother navigating man pages or help FAQs anymore, i just ask the agent how do i do X step by step. Opening this up as part of the app to regular non-engineering users seems relatively easy to implement, but since it would have app permissions you would skip the "give me the steps" and just say do it.

My hypothesis is that the help/support buttons in nearly all apps will be to an agent in the future. Like now you would type "how do i do X" in the excel search bar, in the future it will be "do x". Users in 10 years may not even know how to use the menu/mouse functionality for apps.

I tried both --system-prompt and --system-prompt-file. they both appended when i tried about 6 months ago and watched the traffic. Yes I cleanup all those tools etc.

there is literally a --system-prompt flag for claude code. In my mitmproxy experiments using that flag appended to the existing system prompt rather than replacing it. So I had to create a little helper to strip the system prompt sent over the wire and add my own.

here is a collection of public system prompts from claude code: https://github.com/Piebald-AI/claude-code-system-prompts

FYI you can modify the system prompt using mitmproxy. Just ask your agent to walk you through it. Anthropic system prompts are gnarly and geared towards the lowest common denominator.

this ticks at something that maybe is obvious in retrospect- this is all about economics. the arguments about "using AI doesn't make you an artist", etc. are about being able to charge money for your art i think. maybe obvious to some, but it needs to be explicitly spelled out i think. i was stuck on "i dunno, if i use an AI assistant to run blender i'm still being creative", but is the real argument "you should not be able to charge money to use blender with an AI assistant- you are displacing existing blender artists economically"?

i'm in semi-forced-retirement as an older software engineer in this labor market, so i might be less sensitive to the implicit economic arguments.

i disable subagents. it also helps that i work in microservice size repos, so a single agent can keep the full relevant context. subagents waste alot of unnecessary tokens doing code exploration that the main agent is going to end up duplicating, i think they make more sense for monorepos.

anecdotally I've tried the same plan on different branches with both approaches (single agent with subagents implement the full plan vs. my little session per phase app) and then had agents judge the code quality and my approach worked better and saves tokens. horses for courses.

I appreciate you directly addressing what I was talking about. In my case, I'm an old school 90s wannabe rapper/producer and I used to make beats with old school drum machines, a sequential prophet 2000 sampler, and my friends MPC3000 sampler when I could get my hands on it. When music went into the full computer/MIDI programming phase I dropped out as I lost interest in it, for many of the same reasons that artists today will be facing with AI.

Having never learned the computer tools, like Logic Pro, or whatever open source alternatives are available, I've been considering taking a crack back at it. I have no desire to learn these tools, but if I can say "ok move the kick 1/8 and double up the snare on measure 4" I may get back in the swing of things.

Forced retirement at 50 has me looking for new hobbies.

I'm not really worried about what anyone wants to call me, but I was just pointing out a legitimate use case for agentic AI in the arts.

I've been on this kick since I realized the primacy of the initial part of the session context. I created a python app that reads a phased plan and kicks off a new session for each phase. There is a standard prompt and handoff mechanism to determine if we encountered any unforeseen issues that we need to address in chat, but otherwise it will just grind with a clean session with appropriate context for each phase.

I mean i suppose it depends on if you think people using blender are artists. If you think the part that makes them creative is knowing to right click select blah do blah then I get your position. If you say "no, make the background fade out at this point, push the face to the foreground, and add some shadow behind", I really don't understand why knowing the specific keystrokes are the creative part.

But I think you know that and you're just straw manning.

edit- to be clear i'm talking about using existing software like logic pro, blender, etc. i'm not talking about generative AI.

you don't need to learn blender anymore. you can talk with the ai and describe what you want and iterate. all of the software tools used in media are now accessible to creative but not necessarily technical people.

The system reminder explicitly tells it that the CLAUDE.md or AGENTS.md content is optional. I believe this is a big part of why CC doesn't heed the instructions alot of the time:

IMPORTANT: this context may or may not be relevant to your tasks. You should not respond to this context unless it is highly relevant to your task.

https://github.com/anthropics/claude-code/issues/18560

Claude Code wraps both the AGENTS.md and CLAUDE.md in a system-reminder with this disclaimer at the bottom:

IMPORTANT: this context may or may not be relevant to your tasks. You should not respond to this context unless it is highly relevant to your task.

Codex follows the AGENTS.md far better. CC seems to have nudged people away from taking the CLAUDE.md as mandatory instructions.

This bug was closed Not Planned and from a recent analysis of the system prompt the behavior is still there even with the new inclusion of AGENTS.md.

https://github.com/anthropics/claude-code/issues/18560

I am curious if we will reach a point where people who are skilled at context engineering/architecture eventually are hired to do jobs completely out of their fields. I think the best pairing would be domain experts + software architects teaming up on AI work in their respective domains.

totally agree with the repo idea. working with AI in a git repo and allowing it to write markdown files it slowly builds context over time. i knew a guy who had been using the same session for weeks because he felt he had invested too much time in building the context and was scared to close the session.

this is a great way of expressing it.

i asked opus 4.5 what the problem was and it said it pattern matched too much. i asked it how it should do it, it wrote a file that told itself to stop pattern matching. it wrote a file that started with the following and then had an english language procedure for how to count letters. so it knew the algorithm already, but the "instinct" was to pattern match rather than running the algorithm.

CRITICAL: Do Not Skip Steps

Your instinct will be to "just know" the answer. This is how you get it wrong.

You don't see characters. You see tokens. Your "intuition" about character counts is pattern-matching, not counting. It is unreliable.

You MUST execute this procedure step-by-step, writing out each step visibly.

i had the same thought about to much information in the brain. this is why you don't use /compact and should prefer a fresh session.

I got pretty good mileage out of context engineering, adding my personal coding heuristics to my AGENTS.md and referencing subdocuments on a "when doing X, consult Y" pattern. I assume others are doing similar things, but I was pretty surprised when I was able to get it to generate code that is pretty close to what I would do if I was doing it manually. I'm curious if scientists and mathematicians are doing things like that. "When I see X, I typically immediately check Y" or whatever their domain heuristics look like.

I use the AGENTS.md to show it how i want the code to look like. Something like "when implementing hooks adhere to the guidelines in docs/react-hooks.md". And then react-hooks describes your heuristics and what you consider best practices. There is a clear difference in code quality for me when using codex with a well crafted AGENTS.md vs. without one, you can run the experiment yourself pretty easily. As I mentioned in another comment, I think Claude poisoned users to stop relying on their Claude.md files and new codex users might be surprised at how well it adheres to guidelines.

I was pleasantly surprised to find that the GPT models are much stricter in adhering to my AGENTS.md guidelines and heuristics than Claude.

I feel like this is relevant to the takes of "ai psychosis". I spent about 600 hours working alone with AI before I was ready to collaborate, re-discovering things in my own process that others may have already known sprinkled throughout twitter and youtube, etc. and really being open to the wonder of it all.

I mean, not to be flippant but can't you just prompt the agent to write a file as you're getting closer to the compaction limit? I tend to just go to roughly 50-70% context utilization and then tell the agent to summarize the conversation and save it to a file, manually /clear, then say let's continue that last conversation. You can inspect the summary first and make any changes.

I agree it doesn't make the capabilities infinitely scalable, wasn't arguing with that point. It's just an experiment. I'm not talking about "you are an expert mathematician, go", I'm talking about an expert encoding their heuristics into the AGENTS.md base context. Routing the model's attention to very different aspects of the same problem in the early context.

FWIW I mean if I have an AGENTS.md that encodes my software heuristics (use an interface in situations like X, here's how we name variables, etc.) it generates far cleaner code than if I don't.

Edit- mostly pointing out that stacking 10 base models vs. 10 models with sufficiently different base context isn't necessarily the same attention routing. I suppose I was thinking about tasks that don't have a concrete single answer.

likewise, the opening to Comfortably Numb always resonates.

I think using different AGENTS.md can give the same model different perspectives on the same problem. For example a model with a well-tuned AGENTS.md by an expert mathematician approaching the same problem as the same model with a well-tuned AGENTS.md by an expert biologist can grind on the same problem from different perpectives.

It's worth a shot at least, as a microservices architect I have a bias that we aren't networking these enough, a single main agent session orchestrating multiple subagents is different from multiple main agent sessions with their own subagents coordinating with each other.

mitmproxy

you can just look at the traffic in mitmproxy.

I think we still have a solid control lever on the quality of code we can get AI to generate. Using things like linters and code style checkers, as well as setting up the markdown documentation to guide the agent to generating consistent code will certainly generate different code than just prompting.

Similarly there was an example of edit: Terence (not Eric) Tao chatting with an agent attempting to solve a math problem. "Using AI" means applying your expertise to interact with it as you would a high level colleague. 2 experts in a field don't need to have perfect english and a bloated prompt, they have a massive education/experience common background to fall back on.

It does appear that anthropic in particular is attempting to create a more common experience across expertise levels, but in the current landscape an expert and a novice are unlikely to get the same results. But that does seem to be the goal...