News for hackers. Stuff that matters.

A comment on An LLM does not need to understand MCP. Parent comment

Re:An LLM does not need to understand MCP by 112233 on Thursday August 07, 2025 @10:31AM (#44824943)
Are there any easy to use inference frontends that support rewriting/pruning the context? Also, ideally, masking out chunks of kv-cache (e.g. old think blocks)?

Because I cannot find anything short of writing custom fork/app on top of hf transformers or llama.cpp

[ Parent | on HN ]
Re:An LLM does not need to understand MCP by diggan on Thursday August 07, 2025 @10:35AM (#44825007)
I tend to use my own "prompt management CLI" (https://github.com/victorb/prompta) to setup somewhat reusable prompts, then paste the output into whatever UI/CLI I use at the moment.

Then rewriting/pruning is a matter of changing the files on disk, rerun "prompta output", create a new conversion. I basically never go beyond one user message and one assistant message, seems to degrade really quickly otherwise.

[ Parent | on HN ]