So the embedding model is a fixed-size view on a arbitrarily sized work history (tool calls, natural language messages)? The model is like a summarizer, but in latent space? And not aimed to summarize, but trained to hold whatever is needed for the agent to be autonomous for longer runs?
Pretty much. It's a fixed-size vector per chunk-- 1024 dims in the case of Voyager Nano. The autonomy part is entirely in how you build the vectorDB and query it, not in the model's training. That's the part I've been focusing on lately. Trying different methods and seeing what gives the best results.
At the moment I wouldn't emphasize "autonomous-ness", there's still a fair bit of human hand holding. But once I get a model on the right path it can switch back to to an old project, autonomously locate and debug 2-week old commits and the context around their development, and apply that knowledge to the task at hand.
It's only been a day but I seeing an improvement from nomite (768dims) to Voayager.
As you build up a "body of work" it gets better at handling massive, disparate tasks in my admittedly short experience. Been running this for two weeks. Trying to improve it.