Is the code available?

I had not thought about sharing it. I rolled my own framework, even though there are several good choices. I'd have to tidy it up, but would consider it if a few people ask. Shoot me an email, info in my profile.

The more difficult part which I won't share was aggregating data from various systems with ETL scripts into a new db that I generate various views with, to look at the data by channel, timescale, price regime, cost trends, inventory trends, etc. A well structured JSON object is passed to the analyst agent who prepares a report for the decision agent. It's a lot of data to analyze. It's been running for about a month and sometimes I doubt the choices, so I go review the thought traces, and usually they are right after all. It's much better than all the heuristics I've used over the years.

I've started using agents for things all over my codebase, most are much simpler. Earlier use of LLM's might have been called that in some cases, before the phrase became so popular. As everyone is discovering, it's really powerful to abstract the models with a job hat and structured data.

The framework is now available on Github:

https://github.com/jacobsparts/agentlib

I'm planning to write a blog post about the larger system when I get the chance.

What tooling did you use to make the agents cross-collaborate?

Python classes. In my framework agents are class instances and tools are methods. Each agent has it's own internal conversation state. They're composable and the agent has tools for communicating with the other agents.

Do you try to keep as much context history as possible when passing between agents, or are you managing context and basically one-shotting each time?

Generally, I keep the context. If I'm one shotting then I invoke a new agent. All calls and responses append to the agent's chat history. Agent's are relatively short lived, so the context length isn't typically an issue. With the pricing agent the initial data has been longer than the context window sometimes, but that just means it needs more preprocessing. Now if there is a real reason that I would want to manage it more actively, I can reach out to the agent internals. I have a tool call emulation layer, because some models have poor native tool support, and in those cases it's sometimes necessary to retry calls if the response fails validation. In those cases, I will only keep the last successful try in the conversation history.

There is one special case where I manage it more actively. I wrote an REPL process analyst, to help build the pricing agent and refine the policy document. In that case I would have long threads with an artifact attachment. So I added a facility to redact old versions of the artifact replacing them with [attachment: filename] and just keep the last one. It works better that way because multiple versions in the same conversation history confuse the model, and I don't like to burn tokens.

For longer lived state, I give the agent memory tools. For example the pricing agent's initial state includes the most recent decision batch and reasoning notes, and the agent can request older copies. The agent also keeps a notebook which they are required to update, allowing agents to develop long running strategies and experiments. And they use it to do just that. Honestly the whole system works much better than I anticipated. The latest crop of models are awesome, especially Gemini 2.5 flash.

Cool! When you say “pricing system”, what is it pricing? Is it determining the price in a webshop? Or for bidding ads or so?

Product prices for thousands of listings across various ecommerce channels.

Funny you mention keyword bids, I use algorithms and ML models for that, but not LLMs, yet. Keyword bids are a different problem and more difficult in some ways due to sparsity. I'm actively working on an agentic system that pulls the big levers using data from the predictive models. Trying to tie everything together into a more unified and optimal approach, a long running challenge that I finally have tools to meet.

Do you have a repo for this? I've thought that this would be a great way to compose an Agentic system, I'd love to see how you're doing it.

Langroid has this kind of design (I’m the lead dev):

https://github.com/langroid/langroid

Quick tour:

https://langroid.github.io/langroid/tutorials/langroid-tour/

Looks great, MCP, supports multiple vector stores, and nice docs! How do you handle to subtle differences in tool call APIs?

Thanks!

Langroid enables tool-calling with practically any LLM via prompts: the dev just defines tools using a Pydantic-derived `ToolMessage` class, which can define a tool-handler, and additional instructions etc; The tool definition gets transpiled into appropriate system message instructions. The handler is inserted as a method into the Agent, which is fine for stateless tools. Or the agent can define its own handler for the tool in case tool handling needs agent state. In the agent response loop our code detects whether the LLM generated a tool, so that the agent's handler can handle it. See ToolMessage docs: https://langroid.github.io/langroid/quick-start/chat-agent-t...

In other words we don't have to rely on any specific LLM API's "native" tool-calling, though we do support OpenAI's tools and (the older, deprecated) functions, and a config option allows leveraging that. We also support grammar constrained tools/structured outputs where available, e.g. in vLLM or llama.cpp: https://langroid.github.io/langroid/quick-start/chat-agent-t...

Love it, I did something very similar, deriving a pydantic model from the function signature. Simpler without the native tool call API, even though occasional retries are required when the response fails to validate. Will have to give Langroid a try.