News for hackers. Stuff that matters.
Re:Greedy AI agents learn to cooperate by lettergram on Monday September 06, 2021 @08:31PM (#28439583)
I didn't use deep learning, but even markov chains can lead to behavior that looks this way.

I managed to create "cooperating" agents via a very very simple model:

https://austingwalters.com/modeling-and-building-robotic-sea...

Effectively, we could show adding this circuit would always be dominate in our simulation. Effectively showing that a single behavior (aggression) could lead to cooperation. I.e. defending ones territory, leads to a cooperation. Genetically this is what our simulations always lead to. The more aggressive other "species" were the quicker and more "robust" the cooperation.

Then of course we translated this "learned" behavior into a robot because that seemed like fun lol

Re:Greedy AI agents learn to cooperate by shoo on Monday September 06, 2021 @09:35PM (#28439907)
> even markov chains can lead to behavior that looks this way.

What a fun project, thank you for sharing it!

re:

> Once we created the creatures, we set up their odors, which enable our creatures to smell them. When we have odors and creatures all we then have to do is build a the control components

This sounds similar to the "collaborative diffusion" approach to pathfinding [1].

> However, it is possible to view the code on the github page.

The link to https://github.com/lettergram/pleuro does not work. Do you still have a copy of the code somewhere?

I'm curious to learn more about how you modelled the control loops with Markov chains. Was the rough idea that there are states "forage", "eat", "protect", and then probability of transitions between states depends upon the simulated creature's current state & sensor information about the environment?

edit: aha, i can see there's a 2018 publication by Gillette et al "Implementing Goal-Directed Foraging Decisions of a Simpler Nervous System in Simulation" [2] that has corresponding open-sourced cyberslug code [3]

[1] Repenning 2006 "Collaborative diffusion: programming antiobjects" https://home.cs.colorado.edu/~ralex/papers/PDF/OOPSLA06antio...

[2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5830682/

[3] https://github.com/derekca/cyberslug

not the same codebase, but same advisor, so the code may be some loose relative. alas, the linked codebase seems rather "academic research quality", in the sense that there there's 5000 lines of netlogo cyberslug code and there don't appear to be any automated tests.

Re:Greedy AI agents learn to cooperate by AndrewKemendo on Monday September 06, 2021 @10:43PM (#28440311)
It seems like there might be a trend to use evolutionary learning (genetic algorithms) in conjunction with RL.

That is what Deepmind did with the recent open-ended play agents [1] in order to create more generally capable agents (among a lot of other specific things). In the DeepMind case the largest variable is the environment. In the Intel case the largest variable is the the humanoid component. However both are using task specificity in some narrow task to then bootstrap a combination function to solve more general tasks.

That's definitely a good approach and definitely has a lot of good history showing it's effectiveness. The reason it's notable is because of how it's acknowledging that ensemble methods really are powerful and (especially with RL) that those are likely the way toward extremely general and flexible agents.

[1]https://deepmind.com/blog/article/generally-capable-agents-e...

Re:Greedy AI agents learn to cooperate by Zenst on Monday September 06, 2021 @05:22PM (#28438247)
Interesting and initial thoughts of a hive-mind type approach do seem to be the net result in which the sum of the collective is enabled to perform tasks no individual could.

One further upside of this approach would in effect allow and open up a way for different AI systems to integrate and collaborate

Re:Greedy AI agents learn to cooperate by hoseja on Tuesday September 07, 2021 @04:00AM (#28442194)
If I, your friendly neighborhood software engineer wanted to do reinforcement training for a simple board game, where would I even start?
Re:Greedy AI agents learn to cooperate by svalorzen on Tuesday September 07, 2021 @04:35AM (#28442422)
I maintain a repository of many implementations of classical (tabular) RL algorithms [1] which you might enjoy playing with when starting out. I use it for both research and for student projects. The advantage of avoiding NNs when starting out is that it is much simpler to inspect the inner workings of an algorithm to see whether it's working or not.

I'm always happy to help if something is unclear or difficult so feel free to open issues there :)

[1]: https://github.com/Svalorzen/AI-Toolbox

Re:Greedy AI agents learn to cooperate by lostmsu on Tuesday September 07, 2021 @07:37PM (#28451159)
Alpha Zero would be a good start (a generalized version of Alpha Go, that started the current AI hype cycle) https://duckduckgo.com/?q=alpha+zero+github&t=ffab&ia=web
Re:Greedy AI agents learn to cooperate by TylerLives on Tuesday September 07, 2021 @07:13AM (#28443333)
Kaggle has competitions for games and many people use RL there. Looking through their solutions might be of use to you.
Re:Greedy AI agents learn to cooperate by stevefan1999 on Monday September 06, 2021 @08:29PM (#28439566)
excuse me if I'm wrong, but this sounds like Ant Colony Optimization to me for some reason?
Re:Greedy AI agents learn to cooperate by ruined on Monday September 06, 2021 @05:23PM (#28438262)
this is it. the singularity. ai has finally surpassed human intelligence
Re:Greedy AI agents learn to cooperate by wongarsu on Monday September 06, 2021 @06:48PM (#28438973)
There are plenty of prime examples of greedy humans cooperating. Corruption, cartels, price fixing, collusion in general, ...

More seriously: one of the major struggles of society is setting up incentive structures that align the interests of the individual with that of the group, so even people acting in their own self interest still contribute to the greater whole. We only managed to go to the moon because we managed to turn an utterly selfish ideological struggle into a space race.

Re:Greedy AI agents learn to cooperate by k__ on Monday September 06, 2021 @08:57PM (#28439708)
At the moment it seems that the best performing countries simply found a way to outsource most of their corruption to other countries.

I don't know how long this will work...

Re:Greedy AI agents learn to cooperate by anigbrowl on Monday September 06, 2021 @10:40PM (#28440286)
Those are all rational actors. Many people are irrationally greedy.
Re:Greedy AI agents learn to cooperate by nickik on Tuesday September 07, 2021 @04:09AM (#28442257)
Nobody is perfectly rational, and even if they were, because incomplete information would still not result in rational action.

This applies for cooperation as well. As soon as you have such an organization the people inside that have the same intensive problem. This equally results in the cooperation itself not being rational.

What forms of cooperation works depends again on how to align interest of the people inside them.

Re:Greedy AI agents learn to cooperate by eevilspock on Monday September 06, 2021 @06:05PM (#28438624)
i see what you did there.
Re:Greedy AI agents learn to cooperate by perl4ever on Monday September 06, 2021 @08:00PM (#28439414)
AI could "surpass" human intelligence without ever achieving it.

Like imagine a self-replicating solar energy collector that outcompetes every natural plant. It wouldn't replace the ecosystem in a holistic sense, just a Darwinian one.

There's no reason dinosaurs couldn't have written better poetry than any creature since.

Re:Greedy AI agents learn to cooperate by beaconstudios on Monday September 06, 2021 @08:13PM (#28439493)
Computers have long since surpassed human intelligence - there's no way I could perform as many FLOPS in my head as even a really old computer!
Re:Greedy AI agents learn to cooperate by perl4ever on Monday September 06, 2021 @09:01PM (#28439728)
Not in the sense that I meant. They haven't eradicated all the niches for human intelligence - as a programmer you might overlook the people still making spreadsheets by hand, but I don't think they are endangered, yet.
Re:Greedy AI agents learn to cooperate by drdeca on Monday September 06, 2021 @09:15PM (#28439801)
A vm inside a vm inside a vm, can’t perform as many operations as the base level machine.

It is not clear that such an old machine is capable of doing “more computation” than your brain does.

I very much doubt that either is capable of emulating the other in real time.