I have earlier argued that the sheer number of minds to be coordinated affects the cost of the effort, for a major part of the cost is communication and correcting the ill effects of miscommunication (system debugging). The Mythical Man-Month Frederick P. Brooks Jr.
At the very least, the swarm would be able to set up a means of claiming avenues of thought, then verifying the solutions (or lack thereof) of other agents.
It's quite clear this process - whatever it may be - works, given the results we've seen in the past few weeks.
Here is OpenAI's blog post about Navier-Stokes; [1] which suggests they used smaller connected swarms given the different prompts, then the findings were used to write further follow up prompts for a new group of agents.
> "Here λ is a parameter measuring how parallelisable the task is. They call it the ‘stepping on toes’ parameter. If λ = 1, you have a perfectly parallelisable task, with no stepping on toes and no efficiency penalty. But in reality λ is usually between 0 and 1 — allowing more people to help, but with diminishing returns. For example, if λ = 0.5 then 100 people working together get as much done as 1 person working for 100^0.5 = 10 times as long"
There are diminishing returns to more agents, but they're faster and can scale further than a single smarter agent - it's not really possible to just give 10k times more thinking budget to a single agent because it runs out of context, so for problems that require that much context you can ask the agents to specialize. Each only has a small view of the problem and they probably duplicate a lot of work and step on each others toes and make arcane bureaucracy but there is literally no other way to scale beyond a certain point
just like people
IMO, it is very interesting and also very alarming. At some point the barrier to the exponential progress is the amount of computation available. I would consider hitting this limit and then likely using the same said swarm to help resolve it via various means as very much fitting under the definition of Ray Kurzweil's singularity.
Also, I would disagree, Fable is a sufficient coder. The average human being better, even if that is true now (and personally I wouldn't agree), the average human is limited in the number of 'human instances' he or she can interface with at one time.
if 1000 fable instances is better than any human at a task, how about 1 million instances running? Much easier than finding and hiring and coordinating 1000 extremely productive and intelligent human beings. Much cheaper too, it appears.