It's amazing that most people here don't understand that AI performance in any one computer game relative to humans is largely irrelevant. A system that can play many games at a mediocre level, but does it without any hand-holding, clever APIs or architecture adaptation is infinitely more impressive than a system that can beat everyone in a specific game with all those things applied.

Remember, most humans are completely mediocre at Chess, Go or StarCraft.

Yes, a human is still needed to decide which approach to use, but we are slowly approaching a world where building an AI becomes easier and faster. It will become absolutely irrelevant that a single AI can not play all games, because whenever a new game comes out, someone will be able to build a superhuman AI on it within a week/month.

The brains of an AI are also transferrable. Built a superhuman AI? Send it to a friend! Compare it to a human, who would need to spend enormous amounts of time to transfer their game-brain to someone else. If you want, you can bundle all those AIs into one and pretend it can play any game.

"It's amazing that most people here don't understand that human performance in all computer games relative to AIs is largely irrelevant"

So far I haven't seen Google sending their AIs to a friend.

https://www.tensorflow.org/hub/

https://github.com/tensorflow/models

"All" neural network models is a stretch. Some researchers, including some that work in the industry, do releast their models. The majority don't.

https://www.analyticsvidhya.com/blog/2019/03/pretrained-mode...

Is it not simply the case that, before AlphaStar, very little money and effort was being put into developing AIs for Starcraft 2?

I'm sure that the AlphaStar effort absolutely dwarfed everything that came before in terms of investment (as I'm sure AlphaGo did for Go as well) but based on my reading I'd also bet that Starcraft has likely seen the most continued AI research effort of any imperfect information, real-time game over the last decade - I think that's also part of what made it appealing for DeepMind over Dota 2 or any of the other options in this class of game.

[1] https://arxiv.org/pdf/1909.02682.pdf [2] https://arxiv.org/pdf/1907.09467.pdf [3] https://arxiv.org/pdf/1906.12266.pdf [4] https://liquipedia.net/starcraft/SSCAIT

Even now, I wouldn't really expect to see most game developers start using the latest machine learning techniques to build their own AI's. Maybe the really successful games with competitive leagues might be interested?

I really hope this will change, though. Skilled AIs could make so many games more interesting and better.

Many online games really suffer from bad multiplayer experiences. Players that drop in the middle of a game, ...

If Overwatch, Destiny, ... had really good AIs, that would make the game more attractive. If a player drops, they could replace it with similarly skilled AI. Instead of waiting for 12 players of the same skill-level, they could just play with some AIs.

They could even make it such, that humans win 70% of the time, and let the AIs just lose more often.

AI development probably needs to be made easier, somehow. Game developers have a lot to do.

This point is being brought up a lot, but I don't really buy it. Yes, there have been instances where the AI being too good discouraged players from playing the game as much, but this almost always happens due to the AI having some inhuman advantage that a person could not really replicate. I think players do want the AI to pose a challenge - that's what a lot of casual PvP games are about. Humans are (were?) the only ones that could offer a fair match against another human.

I will absolutely grant you the computational power point though. However, as hardware advances, this cost will become more and more acceptable.

I don't expect to see widedspread adoption of this in commercial games within a decade, but I think that in 2 or 3 decades this will be the norm. In fact, I would bet that once we can make an AI that is good at a game, we can also make it weaker. A game could estimate a player's skill rating silently and then adjust the AI's strength to give the player a difficult/fun time.

Of course, it could just be that AlphaStar is able to play well against humans, because players treat it like a human. Maybe the AI still has standard game AI-like weaknesses that can be exploited if people know that they're playing against an AI. Eg some Diamond league player went mass ravens and kicked AlphaStar's ass. The AI would have to learn how to deal with stuff like this on the fly and I don't think we're there yet.

Why does this matter? Because the trick is often less effort than the real deal. The point of videogames is to entertain you, and the goal of game publishers is to turn a profit while doing so. Obviously there are some exceptions, and indie/free games might have different goals (we all know indie games that can be punishing, like Dwarf Fortress).

I remember with the original Left 4 Dead by Valve, they claimed there was an "AI Director" that modified the map according to how well you were doing, "adapting" to how you were playing. In practice it amounted to closing some sections of the map according to how easy you were killing zombies without dying. So this wasn't really AI but a glorified IF-THEN-ELSE trick, but what does it matter? It served its purpose. Do we really need something more complex (and expensive!) when killing zombies in L4D?

Throwing people into the deep end of the pool usually doesn't work so well. Scaling down the difficulty helps, but really what you want is challenges that teach you something. And you don't really need AI for that.

Last year I started learning to play accordion out of a method book. It has a bunch of songs ordered from easy to hard, and they're also chosen to teach certain skills. It reminded me quite a bit of good level design.

Just look at the achievements for Left for Dead. Only 40% of all players have got the achievement for "kill one of each Uncommon type" which is trivial to get if you play for a few hours.

Just today I saw something that looked interesting, and it was 85% off because of a Halloween sale. So I grabbed it for a couple bucks. Free time being such a rare commodity, there's a solid chance I never even install it... and then there's always Stellaris or Mount & Blade or Medieval Total War that I could happily play until the heat death of the universe, competing for my attention.

Most people who buy your game aren't going to invest even 10 hours in it or come close to maxing out the AI because they only picked it up because it was 85% off during a Steam sale.

Maybe you'll be able to figure out from Early Access numbers how popular your game is and whether you need to invest in super-awesome AI. But it seems like for most games it would be a waste of effort. More likely it would be retrofitted into an already popular game in a DLC or future patch or something.

Maybe the reason people quit your game is because of the AI. What made me stop playing Heroes of Might and Magic 6, Civilization 6, and Total War Three Kingdoms is the poor AI. It takes the fun out of the game, because at lower difficulties the AI puts up no challenge and at higher difficulties you're just exploiting the AI the same way over and over again to keep up.

You might think that that's fine, because you already got the money, but then you'd be looking at it from the perspective of singular games. But studios usually don't just release one game and then disappear. After my experience with HoMM 6, I had zero interest in HoMM 7. 2K Games could release Civilization 7 tomorrow, but without them showing that the game's AI is much better than in civ6, I probably wouldn't pick it up. Yes, the Civilization series will still have players, but it could easily just fizzle out like many other franchises and even genres.

You also can't add this in as a DLC, because people make up their mind about a game near the start. This only works if the game already has longevity, but at that point the game is already popular, so does a better AI at that point make a difference?

Yes, and the current game AIs are completely and utterly incompetent at it. AI in video games only provides a challenge when the mechanics of the game are challenging or the AI is cheating.

>Do we really need something more complex (and expensive!) when killing zombies in L4D?

Not in L4D, because you don't play the game to kill zombies. You play the game to play with other people. If L4D didn't have coop then the game would likely not have gained popularity.

You're right that players want a challenge that they can overcome, but current AI in video games does not offer that. The AI always ends up being so weak that play against the AI cannot be the focus of the game. The focus has to be in mechanical play, puzzles, story, atmosphere, multiplayer or something else. The only way AI keeps up is with inhuman mechanical play or cheating. Neither feels fun to play against, because the counter to it is to figure out how to exploit the AI, after which the game becomes more or less trivial.

I wouldn't say "incompetent", but yes, the mechanics of the game must be challenging and/or (often) the AI must cheat. I'm saying this is enough most of the time; there's really no incentive to build a better AI for videogames because (my hypothesis) most gamers don't care. The illusion of challenge is enough, most of the time. The goal of videogames is to entertain, not to really challenge (again, with the exception of tournaments and competitive gamers, who are a niche within a niche).

You mentioned Heroes of Might & Magic in another comment. I played the heck out of HoMM2, some of (I think) HoMM3 and got bored with the rest. But I wasn't bored with the AI; I got bored because it was the same formula again and again. The genre itself soon felt like a tired formula. Note that the "AI" -- cheating or not -- of HoMM felt horribly difficult to me. I lost battles more often than I won; the only trick that reliably worked was to start the fight with a lot more troops than the computer enemy, and that's not much of a trick! Not a lot of places to learn effective tactics back then either, maybe the hellpit that was GameFAQs?

The article describes "exploiter agents that focus on helping the main agent grow stronger" as key to their approach. That seems promising? Maybe it could be used to make bosses with specific strengths and weaknesses where part of the game is figuring out how to beat them.

Chess and Go are already quite different from NLH, because they are purely about strategy. The best strategy in Chess either wins or leads to a draw if played by both sides every game. In NLH an optimal strategy just breaks even (ignoring the rake) against other optimal players and makes money on average against anybody else. But over even a few hundred hands you can't tell.

Which is why the most complex game of all is actually intergalactic horseshoes.

Even basic strategies will win if they’re done faster. APM (actions per minute) is a very significant factor into who is winning. Apparently they limited their AI player to 264 APM but that’s still incredibly high and done with machine level consistency. That’s almost 4.5 actions per second!! I know there are human level players at and probably above that level but that really allows for basic strategies to win out.

This isn't really true. Basic strategies done faster still lose miserably to humans, we can see this from the long history of SC Broodwar AI tournaments where they have a human play the best AI at the end (and always win).

Faster helps, better strategy helps more, maybe we can't say the AI is doing as well as the top humans at strategy since it's faster, but we can say that it's doing pretty good.

    AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE)
    IEEE Conference on Computational Intelligence and Games (CIG)
    Student StarCraft AI (SSCAI) Tournament
    BWAPI Bots Ladder

A great example would be seeing the faint image of cloaked units. How does that work with the AI? Can the just phrase the current screen and instantly see any cloaked unit? I can imagine an attention system where more attention dedicated to part of the screen would increase the probability of noticing a cloaked unit there.

Actually 792 APM. They limited their AI to 264 "things" (choose unit+ability+target or change view) per minute. Each "thing" can be counted as between 0-3 actions by starcraft. So it really has a peak APM limit of 792.

From the article:

>Agents were capped at a max of 22 agent actions per 5 seconds, where one agent action corresponds to a selection, an ability and a target unit or point, which counts as up to 3 actions towards the in-game APM counter. Moving the camera also counts as an agent action, despite not being counted towards APM.

But from replays it has an average APM of 248 at the highest, so nothing crazy going on. Plenty of pros have peak APMs during fights that are higher than 800.

I don't think it's like that at all. On the high level, there is no "chess AI", "go AI", "image classification AI" and "dexterous manipulation AI". These are all sides of the same coin, that gets significantly better every year. Adding support for the new game or new "environment" to existing deep learning based backbone still requires a bit of engineering work and a few creative tricks to unlock the best possible performance, but the underlying fundamentals are already there and are getting better and better understood.

There is a reason why the progress in AI is so hard to measure. Anytime a next task is solved, there is a crowd saying it's not a "real AI" and that scientists are solving "toy problems". Both statements are totally true. But the underlying substance is that each of these toy problems is of increasing complexity and brings us closer and closer to solving the "real problems", which are mostly so undeniably complex that we couldn't attack them upfront. Still, the speed of progress in the field of AI research is staggering and it's hard to keep up with it even for professional researchers who spend all their waking hours working on these things.

6 years ago we were able to solve some Atari games from pixels. Today, that feels like a trivial exercise compared to modern techniques. With billions of dollars of investment pouring in and steady supply of fresh talent, it is very hard to predict what the pace of research will be in the coming years. It is entirely possible we'll encounter a wall we won't be able to overcome for a very long time. It is also possible that we won't, and in that case we're in for a very interesting next few decades.

On a practical level, this is not true. There are different algorithms, different architectures, different hyperparameters required for each of these problems, and often for each subdomain within each of these problems, and often for each specific instance of these problems. It's difficult to draw any kind of holistic picture that combines all of the individual advances in each of these problem instances; that's why progress in AI is so hard to measure, and why a statement like "each of these toy problems...brings us closer and closer to solving the 'real problems'" is probably a bit too coarse-grained to be fair as well.

Deepmind's best-in-class chess and Go AIs are the same code (AlphaZero) just given respectively rules and game state input for either chess or Go and then allowed to train on the target game.

One of the fun works in progress in this space is teaching AIs to play a suite of 80s video games. Getting quite good at several games where the idea is to go right and not die is pretty easy these days, but Deepmind's work can do a broader variety only coming badly unstuck on games where it's hard to discern your progress at all without some meta-knowledge.

You claimed that "different architectures" are needed. Not true. And further you claimed this is true even for "each subdomain". This would have been a fair point in 1989. Traditional chess AIs approach the opening very differently for example, relying on fixed "books" of known good openings. But AlphaZero since it is a generalist doesn't do this, it plays every part of a match the same way.

Now you've gone from asserting that Chess and Go need separate AIs to claiming that since BERT and AlphaZero are different software it makes your point. Humans pretty clearly don't have a single structure that's doing all the work in both playing Go (AlphaZero) and understanding English (BERT) either - so that's a pretty bold bit of goalpost moving.

> Anytime a next task is solved, there is a crowd saying it's not a "real AI" and that scientists are solving "toy problems". Both statements are totally true. But the underlying substance is that each of these toy problems is of increasing complexity and brings us closer and closer to solving the "real problems"

I wonder if this is true. This belief may seem like common sense, but it's not obvious to me that domain-specific problems must generalize to General AI ("real problems") or even bring us closer to it. That is, it's not evidently true that many small problems will eventually lead to a general solver of everything (or to human-like intelligence). Or to say it in yet another way, it's not obvious to me that human-like intelligence is the sum of many small-problem-intelligences.

Again, common sense may lead us to believe this, and maybe it's true! But I think this conclusion is far from scientifically evident.

You can even interleave the training for the second task with a few training rounds for the first task to maintain proficiency. There's a group that's using this sorry if technique to make a general "plays videogames" AI. I couldn't find a good link from my phone, but here's a less good link about something similar: https://towardsdatascience.com/everything-you-need-to-know-a...

As another poster said these are all tasks performed by different systems. For chess and Go AI it's Deep Reinforcement Learning with Monte Carlo Tree Search. For image recognition it's Convolutional Neural Networks. Importantly, these systems are very task-specific. You won't find anyone trying to beat humans at games using CNNs, for example, or using Deep-RL to do text recognition. Far from "a few creative tricks" these are systems that are fundamentally different and are not known to generalise outside their very limited domains. They're one-trick ponies.

The OpenAI paper on "dexterous manipulation" reported learning to manipulate one cube, the same cube, always, after spending a considerable amount of resources on the task. It was a disappointing result that really shouldn't be groupwed with CNNs and Deep-RL for game playing. The level of achievement does not compare well.

>> Anytime a next task is solved, there is a crowd saying it's not a "real AI" and that scientists are solving "toy problems".

This used to be the case a decade or more ago. In the last few years the opposite is true. The press is certainly very eager to report every big success of "AI"- by which of course is meant deep learning.

>> 6 years ago we were able to solve some Atari games from pixels. Today, that feels like a trivial exercise compared to modern techniques

6 years ago DeepMind showed superhuman performance in seven Atari games with Deep-RL (DeepQN in particular): Beam Rider, Breakout, Enduro, Pong, Q*bert, Seaquest and Space Invaders. Since then more Atari games have been "beaten" in the same sense, but many still remain. I'm afraid I can't find references to this but I've seen slides from DeepMind people a few times and there is always a curve with a few games at the top and most games at the bottom, below human performance. There are some games that are notorious for being very difficult to solve with Deep-RL, like Montezuma's Revenge which was claimed to be solved by Uber a couple of years ago however this was done using imitation learning, which means watching a human play. The result is nothing like the result in Go, which remains the crowning achievement of Deep-RL (and its best buddy, MCTS).

Bottom line: Atari games remain anything but a trivial exercise.

And the architectuers that play Atari do not perform as well in Go or chess, say. You are mistaken that it's simple to train the same system to do all of those things. The AlphaZero system that played Go, chess and Shoggi well enough to beat its predecessor (you will excuse me that I don't remember which incarnation of Alpha-x it was) had an architeture fine-tuned to a chessboard and pieces with discrete moves, so it would not be possible to reuse it to play Starcraft, say, or even tic-tac-toe. The cost to train AlphaZero is also very high, in the hundreds of thousands of dollars.

In pretty much any field, top performing humans are at the physical limitation level, you will not see any sort of breakthrough, just incremental improvement.

Machines on the other side, can be scaled arbitrarily. Once you've built a small crane, you can build even increasing ones, it's just a function of money and interest.

Some say that intelligence is not like that, that it can't be scaled arbitrarily. But the burden of proof is on them, especially given the consequences if it can.

It doesn't matter how much money or interest we have, but right now it isn't technically feasible to build a 36000 km tall crane (also known as a space elevator). Humanity simply couldn't get it done even if we poured all our current resources into that project. It isn't physically impossible, but such a behemoth has requirements that current materials science cannot meet. Building tall cranes generates useful know-how for a space elevator, but it's a really different problem, not just a matter of resources.

Following the analogy, a general purpose AI simply isn't a bigger Deep Blue or AlphaGo; it's probably something different that requires knowledge that we currently don't have. Sure, building Deep Blue and AlphaGo most likely generated part of that money, but that doesn't mean we have everything we need.

You can argue that is simply also money and interest, and that's true, but it's not in the same as building a 10 meter crane and a 20 meter crane.

I agree with this, but it isn't clear to me that a general AI will have a significantly different impact on society than a world where task-specific well performing AIs are easy for anyone to develop.

Sure, a general AI has a set of properties that are really fascinating to discuss and debate (including what is consciousness and whether AIs should be given rights), and perhaps a general AI is required for doomsday computers-taking-over scenarios, but the impacts that AI will have on our economy and politics don't require general AI.

However, I disagree that the impact is the same. I think that the difference between a lot of specific jobs being automated and the world being run by a MULTIVAC world optimizer is pretty big.