https://github.com/awesoftsolutions/google_gemini_flash_comparison
We ran seven Google models through the same task, in the same multi-agent harness.Improvements to reasoning and tool calls are the main reasons for the improvements. We saw less and less reasoning and some of the lowest invalid tool calls across the entire evaluation suite.Github link for the run output and analysis. More at evals.favur.dev
Then that TOC becomes the "required read" before everything in any new agent. Which increases the memory doc read substantially since it understands what each doc is natively.
Curious what made you take it - was it mostly about how the thing feels to use day to day, or something you kept hitting with the Electron ones?
https://github.com/awesoftsolutions/idea_meta-muse-2048-game
We provided our harness with a single objective - 2048 with a twist - and 2 days later it delivered.https://github.com/awesoftsolutions/idea_meta-muse-2048-gameThe run is recorded end to end, so the code can be checked against the process that produced it. Our app is drivable via replay here: https://favur.dev/go/drive/the2048
Maker here. Favur is a multi-agent harness written in Python: a team of 14 specialized agents — planner, architect, coder, tester, reviewer, builder among them — coordinated by Favur itself, not by an LLM. It takes a written statement of work and produces completed code, with no hand-holding or constant guidance along the way. Favur Evals scores those runs across models, using the same...