A situation to think about: if I were to write a rule that said "I am using Tailwind CSS for styling", then this is actually information that can't be just known. It's not "common knowledge", but instead "preference" or "personal knowledge". I do think it's a fair response to say "can't it just read my package.json"? Probably this works in a handful of cases, but I've come to find that even so there are a few benefits to custom rules that I expect to hold true regardless of LLM progress: - It's more efficient to read a rule than to call a tool to read package.json on every request - Especially in large enterprise codebases, the majority of knowledge is highly implicit (oftentimes in detrimental ways, but so the world works)
But yeah this is a majorly important and interesting question in my mind. What types of customization will last, and which won't? A blog post would be amazing
Can you please share what are your favourite tools and for what exactly? Would be helpful
I've been using Cline a lot with the PLAN + ACT modes and Cursor for the Inline Edits but I've noticed that for anything much larger than Claude 3.7's context window things get less reliable and it's not worth it anymore.
Have you found a way to share knowledge packs? Any conventions? How do you manage chat histories / old tasks and do you create documentation from it for future work?
I wrote my own open source one here: https://github.com/drivecore/mycoder. Covered on hacker news here: https://news.ycombinator.com/item?id=43177117
I've also studied coding with it and wrote a lot about my findings here:
- https://benhouston3d.com/blog/lean-into-agentic-coding-mista...
- https://benhouston3d.com/blog/building-an-agentic-code-from-...
- https://benhouston3d.com/blog/agentic-coder-automation
- https://news.ycombinator.com/item?id=43177117
- https://benhouston3d.com/blog/the-rise-of-test-theater
My findings are generally that agentic coders are relatively interchangeable and the reason they work is primarily because of the LLM's intelligence and that is a result of the training they are undergoing on agentic coding tasks. I think that both LLMs and agentic coding tools are converging quite quickly in terms of capabilities.
> Have you found a way to share knowledge packs? Any conventions? How do you manage chat histories / old tasks and do you create documentation from it for future work?
I've run into this wall as well. I am working on it right now. :) Here is a hint of the direction I am exploring:
https://benhouston3d.com/blog/ephemeral-software-in-the-era-...
But using Github as external memory is a near term solution:
https://benhouston3d.com/blog/github-mode-for-agentic-coding
It would be great to run local models at the same level one day. I am sure that Claude is making your wallet feel quite light :)
https://benhouston3d.com/blog/building-an-agentic-code-from-...
I'm particularly fascinated by those last two links, along with your latest post about READMEs. It makes me wonder about a visual specification editor that provides GitHub-like task chronology around the spec, with the code as a secondary artefact (in contrast to GitHub, where code is primary).
BTW I'm working on ideas to make code a secondary artifact right now, sort of v2 of mycoder.
Fantastic!
Agentic task-offloading is already here; it is just unevenly distributed.
> Basically knowledge packs should be specified in each npm package.json or similar.
Our YAML-based file format for assistants (https://docs.continue.dev/reference) is hoping to do just this by allowing you "import knowledge packs".
Does it need to be decoupled from package.json, etc.? One of the most interesting reasons we decided not to go that route was it can be cumbersome for all of your many dependencies to taken into account at once. Another is the question of how the ecosystem will evolve. I definitely think that each package author should take the time to encode the rules and best practices of using their library, however it might be difficult for community to help out if this is gated by getting in a pull request.
At the same time, one of the soon-to-be-released features we are working on is the ability to auto-generate or suggest rules (based on package.json, etc.).
Knowledge packs should be decoupled from package.json just as eslint rules or commit-lint rules are. You can include them in package.json or in separate files. But including pointers to the main files in a package.json helps with discovery.
All packages across all languages should support AI friendly knowledge packages so a level of decoupling is required.
EDIT: After thinking about it I think README.md should just be written with Agentic Coders in mind. I wrote up my thoughts on that here: https://benhouston3d.com/blog/crafting-readmes-for-ai
> consider letting the LLM write it's own "Readme" for a package on the hub be scanning source/types of the package
This is something we're looking to ship soon
Basically knowledge packs should be specified in each npm package.json or similar.
I guess I should write this up as blogpost or something similar.