Yes, this is exactly what I did! I directed the high-level architecture and structured the optimization tasks, but for the "inner loops" of e.g. precisely matching the filter response, I set the agent in goal mode and let it take control of the MIDI and audio interfaces