So we agree on a message system having potential. But why the vectors? In any case, interesting stuff.
I'm using vector embeddings for creating code views based on semantic search, initially based on the user prompt. That really works wonders to give the agent a flying start.
> allows agents to request (and later on drop) 'views' on functions and other logical pieces of code [...] The problem is that the models are not trained for this
Fwiw, I was playing with an "outliner"-tool collapse/expand idiom, on synthetic literate-programming markdown files, with #ids on headers and blocks. Insufficient experience to suggest it works, but it wasn't obviously not working, and that with a non-frontier model and very little guidance. Other familiar related idioms include <details>/<summary>, hierarchical breadcrumbs, and plan9-ish synthetic filesystems `foo.c/f.{c,dataflow,etc}`. One open question was comfort with more complex visibility transformations or sets - "hide #bar; show 2 levels of headers-only under #hee; ...". Another was cleanup - recognition of "I no longer need this and that".
The problem is that the models are not trained for this, nor for any other non-standard agentic approach. It's like fighting their 'instincts' at every step, and the results I've been getting were not great.