The current series of prompts I'm using is somewhat 'manually manage/allocate memory' of the LLM context window.
It starts with a PROMPT.md with this content.
```
create or update implementation_status.md with the implementation status
study /specs and implement what has not been implemented yet in /src
create property-based tests inline in the source file that is being implemented
run "make" to verify the implementation after each change
```
If it goes off the rails (and it does) - restart the chat and use this.
```
@prompt
continue
```
None of this works without applying /stdlib to control the technical outcomes/patterns to steer the LLM. Otherwise you just get slop.
The LLM coding tool I have the most experience with is Windsurf IDE + Sonnet 3.5. In Windsurf you can define both global and project rules in .md files. I have found that managing those files very closely is key to success. They just tried to automate that with auto-generated "Memories" but they are generally slop, and I delete them. Managing the project rule file will save you so much pain. That is where I define the frameworks and APIs to use.
Prompting to check the schema is great, but prompting against live data in the DB is amazing! For example:
> When logged in as user [email protected], I can't see the Edit button on src/pages/projects/show.tsx. Please use MCP to check if I have the correct permissions in organization_roles. If I do, look at the UI code to see if we are checking it correctly."
Wowzers!
> We evaluate 12 popular LLMs that claim to support contexts of at least 128K tokens. While they perform well in short contexts (<1K), performance degrades significantly as context length increases. At 32K, for instance, 10 models drop below 50% of their strong short-length baselines. Even GPT-4o, one of the top-performing exceptions, experiences a reduction from an almost-perfect baseline of 99.3% to 69.7%. [0]
I copied all the code that I could from TFA and pasted it into OpenAI's tokenizer. It counted ~15k tokens. Many other tokens were generated in TFA's chat, some of which are not visible to the user. I think it's fair to assume that the entire chat was at least 25k tokens, right? Therefore, I believe that by the end of that chat, 4o's performance was significantly degraded.
I think a major skill to develop for LLM supported coding is to compress a chat after just few thousand tokens into something like a step1.md file. Then start a new chat with "read step1.md" as the first prompt, and so on..
Is my logic sound here?
[0] https://arxiv.org/abs/2502.05167