News for hackers. Stuff that matters.
Re:Project Sid: Many-agent simulations toward AI civilization by hackathonguy on Sunday November 03, 2024 @08:22PM (#42037574)
I'm curious if it might be possible that an AI "civilization", similar to the one proposed by Altera, could end up being a better paradigm for AGI than a single LLM, if a workable reward system for the entire civilization was put in place. Meaning, suppose this AI civilization was striving to maximize [scientific_output] or [code_quality] or any other eval, similar to how modern countries try to maximize GDP - would that provide better results than a single AI agent working towards that goal?
Re:Project Sid: Many-agent simulations toward AI civilization by wombatpm on Sunday November 03, 2024 @09:20PM (#42037827)
Paperclip production?
Re:Project Sid: Many-agent simulations toward AI civilization by bbor on Sunday November 03, 2024 @08:31PM (#42037614)
Yes, good sense for progress! This has been a central design component of most serious AI work since the ~90s, most notably popularized by Marvin Minsky’s The Society of Mind. Highly, highly recommend for anyone with an interest in the mind and AI — it’s a series of one-page essays on different aspects of the thesis, which is a fascinating, Martin-Luther-esque format.

Of course this has been pushed to the side a bit in the rush towards shiny new pure-LLM approaches, but I think that’s more a function of a rapidly growing user base than of lost knowledge; the experts still keep this in mind, either in these terms or in terms of “Ensembles”. A great example is GPT-4, which AFAIU got its huge performance increase mostly through employing a “mixture of experts”, which is clearly a synonym for a society of agents or an ensemble of models.

Re:Project Sid: Many-agent simulations toward AI civilization by ValentinA23 on Monday November 04, 2024 @08:06AM (#42041160)
I don't think "mixture of experts" can be assimilated to a society of agents. It is just routing a prompt to the most performant model: the models do not communicate with each other, so how could they form a society ?
Re:Project Sid: Many-agent simulations toward AI civilization by bbor on Tuesday November 05, 2024 @11:10AM (#42052613)
Hmm that's a good point, but IMO the distinction isn't sharp enough to make a big deal over. The core idea of SoM as I see it is that human cognition is often quite decentralized, and that any illusion of a unified self is constructed piecemeal from the outputs of smaller, less-aware subsystems. Generally it's expected that the subsystems communicate with each other, yes, but I think "disproportionately rely on one or two members for complex questions but act like you're unified overall" still fits the bill.
Re:Project Sid: Many-agent simulations toward AI civilization by ValentinA23 on Wednesday November 06, 2024 @09:57AM (#42063271)
The opinion I formed during the first few months of GPT4 release was that the society of the mind hypothesis was being disproved by the "maximalist" approach some were undertaking in order to build a true AGI. Turned out composing many LLMs into a cognitive architecture where each one had a specific purpose (memory, planning, etc ...) wasn't scaling.

On the same note, I suggest the following: training a transformer by "slicing" it in group of layers and force it to emit/receive tokens at each of those group's boundaries. What I expect: using text rather than neural activations should lead to decreased performance.

This is something you can observe in our societies: intelligence doesn't compose, you just don't double a group's overall intelligence by doubling the number of members. At best you'll observe decreasing return, at worst intelligence will decrease.