Code: https://github.com/facebookresearch/diplomacy_cicero

Blog: https://ai.facebook.com/blog/cicero-ai-negotiates-persuades-...

Site: https://ai.facebook.com/research/cicero/

Expert player vs. Cicero AI: https://www.youtube.com/watch?v=u5192bvUS7k

RFP: https://ai.facebook.com/research/request-for-proposal/toward...

"In a game in which dishonesty is commonplace, it is notable that we were able to achieve human-level performance by controlling the agent's dialogue through the strategic reasoning module to be largely honest and helpful to its speaking partners."

This is one of the key insights in playing Diplomacy well. You can turn honest opposition in the early game into trust, which is helpful for flipping allegiances in the late game. ('Yes, I've been trying to murder you the entire game, but I've been honest about it, and never lied to you once. Now I see that we've got a real opportunity to dislodge Russia and sweep the eastern half of the board if we work together... What do you say?') Dishonesty means people don't want to work with you, because you are not a dependable partner. So you gotta make sure that any lies you tell pay off /big./

Well, yes, "largely" honest, until the critical moment when you stab to take 18 supply centers to win the game.

You don't win Diplomacy by staying honest, you win by choosing the right moment to make the move and burn that trust. It's the 5% dishonesty that matters.

It's interesting and I think significant that people are uncomfortable with positive values, such as honesty, doing good for others, responsibility to others, right (not wrong), etc. We shoot them down, similar to how people instinctively cut off a conversation going in an uncomfortable direction, often without even realizing it.

I can't attribute that the parent - I don't know the individual - but I expected a chorus of such responses. I feel like I can see them squirming in their seats ...

But I feel it too. There's societal pressure against doing good. It's often more socially comfortable to throw a cigarette butt on the ground than to pick one up and discard it. That's bizarre, it's perverse incentives, and we can and should change it.

It's perfectly reasonable to advise against good values in a game that's won by people who strategically abandon theirs. If you want to win, that's the strategy to follow.

Not necessarily a lesson for life unless you view life as a finite game to be won. But in that case, it's probably the best strategy as well.

I don't view life like that and I'm guessing you don't as well, but I'd advise anyone to be cautious of their interactions with people who play to win.

When people are concerned about AI alignment, they are not concerned about an AI that is a pathological liar, lying for no reason.

They are concerned about an AI which is believable day-to-day but is secretly optimizing for its own goals, which are only exposed when it eventually makes a deceitful move it cannot disguise, but succeeds because it now has sufficient control.

That's literally what the AI is doing here; being honest until the moment it is able to stab everyone else and throw them under the table.

I think what you’ve identified is a societal immune response to predictable morality being systematically exploited by amoral actors to gain wealth and standing.

I also think there is a big gap between what most people advise and what they ultimately do that biases toward the right thing. That is, people indicate they will not be moral in order to dissuade others who might try to exploit it, but in the end they generally behave morally.

Personally I will advise other people to be cautious while repeatedly leaving myself open to being taken advantage of. It almost never happens, and you learn to identify those who will.

Of course this all pivots when/if you enter corporate leadership and part of your job is to use the morality/comfort of others as part of the container for getting your role accomplished.

> I also think there is a big gap between what most people advise and what they ultimately do that biases toward the right thing. That is, people indicate they will not be moral in order to dissuade others who might try to exploit it, but in the end they generally behave morally.

That is very interesting. Have you seen any research on it?

> while repeatedly leaving myself open to being taken advantage of. It almost never happens, and you learn to identify those who will.

My thinking: There is no perfectly safe solution. People who think I'm taking naive risks don't get such great results themselves. I think being 'open' is generally safer - humans generally follow the lead of those around them, and I get better responses.

> you learn to identify those who will.

Yes, it cannot be overstated: Being honest, you develop expertise in the skills of executing honestly, and as with any skill that expertise enables you to evaluate those skills in others. Do otherwise, you acquire other skills.

> That is very interesting. Have you seen any research on it?

The relevant area of research I’m familiar with is around dishonest signals (and then costly signals) in evolutionary theory.

So Diplomacy is just an acrimonious version of iterated prisoner's dilemma?

Demis Hassabis had better watch out! (Besides being CEO of Deepmind he is also a top Diplomacy player)

dup of https://news.ycombinator.com/item?id=33706750

Oh crap how did we miss that. Comments moved thither. Thanks!

zzzz

Great, let's definitely teach computers to play a game that requires strategic thinking, a robust theory of mind, the capacity to lie convincingly to your enemies, and well-timed betrayal, with the goal of world domination. Can't see any ways for that to go wrong.

One more: it’s Facebook or all places teaching them

How else would Mark have someone to hang out with in the metaverse?