drphilwinder

I appreciate the discussion. Thanks.

Thanks. What do you mean by faster here? Wall clock time?

I can't.see myself specifying 60 bits of work to work on in parallel. So I guess you're doing some kind of task decomposition.

But I'm not convinced that this is that much more efficient, given the complexity of the coordination overhead.

E.g. in my mental model, 3 big tasks takes about the same amount of time as 60 small tasks after planning and coordination.

What's your experience?

Thanks for sharing. Really interesting.

At what point do you think you tip over from getting away with a handful of agents working on different things in one repo, to needing a comms/coordination system for your tasks?

I think there must be a burden to run this and I'm guessing it only pays off when you get to some kind of scale. What is that scale? Other human team members? Sheer workload?

This is a great point. Not that far. We also snapshot the desktop for "slow" non-streaming updates to the UI. We could push these into Claude itself to act on or describe or whatever.

Hi all. This is an announcement post for a project I'm looking to get very early feedback on.

I've been using an AI coding assistant for a while (Cursor, Cline, etc.) and found that quite a few problems are caused by the model not having up to date or relevant examples of problems I'm working on.

So I created Kodit, an MCP server that aims to index your codebases and offer up relevant snippets to the assistant.

This works well when you're working with new projects, private codebases, or updated libraries.

I'm launching now to get as much feedback as I can, so do give it a try and let me know what you think!

- [Docs](https://docs.helixml.tech/kodit/) - [Repository](https://github.com/helixml/kodit/) - [Announcement post](https://blog.helix.ml/p/helix-kodit-open-source-mcp-server?r...)

For me, one of the most important parts of the spec is formulation of the DAG of tasks. Whether that be calling other LLMs or some retrieval mechanism.

What does everyone think about the pros/cons of a formal DAG specification vs using natural language? E.g. defining the DAG in yaml vs. something more natural language like writing logic and making calls in the prompt text?

Check out https://unify.ai/chat if you're interested in a router optimised for cost/ttft/performance for commercial language models.

Under the hood right now it's on-prem llama3 + a pretty basic RAG pipeline. The coolest thing about this technically is that it's all running totally privately.

But the main goal is to make websites more efficient. To get your customers to the answer they need faster.

Your sentiment is correct, but it's more of a spectrum. Fine tuning can learn facts (otherwise how would the foundation models learn facts?). But it needs those facts in the training dataset. If you have an infinite amount of facts, then you can memorise all of them.

The challenge arises when it becomes hard to generate that training data. If you just have the raw text and pop that in the context (i.e. RAG), then the LLM can be just as factual without any of that hassle.

Q2: identifiers in the prompt to say "you've been trained on this, only answer questions about this".

Q3: Depends on the size of the training data/docs. For the average PDF, about 30 minutes.

Give it a try!