News for hackers. Stuff that matters.
Re:Handbook.md shows that long policy documents do not relia... by spIrr on Wednesday July 29, 2026 @09:45AM (#49097464)
As a hobbyist, I find it difficult to figure out how to make Claude stick with some repeating things I want it to do after every major action, like re-evaluate the completeness of tests, update the documentation, etc. And CLAUDE.md/AGENTS.md definitely did NOT help there, sadly.
Re:Handbook.md shows that long policy documents do not relia... by victorbjorklund on Wednesday July 29, 2026 @10:07AM (#49097754)
Hooks can be pretty useful for that. A hook when it is finished ”run tests suite and check coverage” ”check if your changes require updating the docs”
Re:Handbook.md shows that long policy documents do not relia... by Muromec on Wednesday July 29, 2026 @10:02AM (#49097675)
Don't use the default harness, write your own instead.
Re:Handbook.md shows that long policy documents do not relia... by TeMPOraL on Wednesday July 29, 2026 @10:14AM (#49097837)
Any way to do that AND use subscription instead of per-token pricing from SOTA providers?
Re:Handbook.md shows that long policy documents do not relia... by Muromec on Wednesday July 29, 2026 @10:30AM (#49098015)
If the harness runs on your localhost, but the inference doesn't, it usually means it's calling some API. Whether you want to break TOS of your provider like that or simply buy tokens from our friends from UTC+8 timezone is an open question.
Re:Handbook.md shows that long policy documents do not relia... by Supermancho on Wednesday July 29, 2026 @10:07AM (#49097756)
This is the way. Making your own agent to have a sticky memory context that is prepended to every execution is necessary to ensure each task is bounded by those precepts.
Re:Handbook.md shows that long policy documents do not relia... by Muromec on Wednesday July 29, 2026 @10:18AM (#49097883)
The trick I'm doing -- the model is given a tool that runs a prompt in the current thread to consolidate it's working memory and identity (it has a memory tool bound to the agent persona). When the prompt ends, the parts of memory that are marked as identity are merged together into a new system prompt, then the context restarts with only system prompt and this tool call surviving. Then it just keeps going.
Re:Handbook.md shows that long policy documents do not relia... by drob518 on Wednesday July 29, 2026 @10:42AM (#49098185)
Doesn’t that blow your token cache hit rate and balloon your costs (essentially everything is billed at the input token rate, not the cached rate)?
Re:Handbook.md shows that long policy documents do not relia... by haus20xx on Wednesday July 29, 2026 @11:29AM (#49098806)
If it moved a block like

<system_prompt>

<tools_etc>

<project_prompts>

<user_call>

<llm_response>

=>

<system_prompt>

<tools_etc>

//Removed project_prompts

<user_call>

<llm_response>

<user_call>

<project_prompts> //reinserted

<llm_response>

Cache would break but if you did instead

<system_prompt>

<tools_etc>

<project_prompts>

<user_call>

<llm_response>

<user_call>

<project_prompts> //Duplicated with new user messsage

<llm_response>

It wouldn't bust cache, it would just make your input prompts slightly larger. Technically inefficient as you're duplicating the same rules over and over but I imagine for a smallish checklist/principles that it is tremendously more efficient than a cache break every message

Re:Handbook.md shows that long policy documents do not relia... by cyanydeez on Wednesday July 29, 2026 @12:41PM (#49099784)
so first, we treat context like a stack. Things are popped onto the stack; if we try to remove anything but the top of the stack, we'll break the cache.

So a good harness should be popping tool prompts, user hints after every return from the top so the context of tools and user hints never repeats and are allways prefixed to the current user prompt.

I've never inspected any harness, but opencode with the dynamic context compaction plugin appears to do this well.

Re:Handbook.md shows that long policy documents do not relia... by Muromec on Wednesday July 29, 2026 @04:03PM (#49102295)
By the time it does 10 turns of tool calls it forgets about the nagging again. It can and does ignore direct instruction from the user prompt too, sometimes repeatedly
Re:Handbook.md shows that long policy documents do not relia... by cyanydeez on Wednesday July 29, 2026 @10:39AM (#49098136)
What I'm currently doing in a large refactor, is I created a super-run script; the super run script is devided into super-dev (Setup dev), super-test (run all tests), super-build (build artificats), super-e2e (test all artifacts), super-deploy (deploy finished).

Each super's sub functions should _fail hard_, and each script should be highly detailed; of course I'm not doing it myself, but in small increments of directed work, it can build up the necessary harness.

What I get is a CI that just starts with "run super-run.sh" and that gives it context, then each sub script provides context depending on if it succeeds or fails. If it fails, the agent is provided what it needs.

It's basically, you have to design the products of the AI to give itself the context. Another technique I'm testing out is a parallel set of files like <subject-module>.js, <subject-module>.test.js, <subject-module>.md which get pulled up if the Agent is looking for a file.