News for hackers. Stuff that matters.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by panic on Wednesday March 09, 2016 @03:06AM (#11251593)
Regardless of how good the deep learning network is on its own, the algorithm described in the DeepMind paper is an improved tree search.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by brian_cloutier on Wednesday March 09, 2016 @03:26AM (#11251669)
Forest for the trees.

Monte Carlo Tree Search was necessary and itself a massive improvement over minimax but not sufficient for creating a Go program to challenge professional players. The true innovation here is the neural networks. Without those networks to guide it AlphaGo plays far worse than existing programs.

The fact that those networks are sufficient is pretty incredible. We already knew that by inventing them we had created a very pure form of pattern recognition, but it's surprising that the pattern recognition coupled with some tree search seems to be all you need to play Go as well as humans.

It's not impressive to you that we've now reproduced a piece of human intelligence, "intuition", which was previously considered out of reach?

Re:AlphaGo beats the world champion Lee Sedol in first of fi... by nazka on Wednesday March 09, 2016 @07:02AM (#11252427)
> but it's surprising that the pattern recognition coupled with some tree search seems to be all you need to play Go as well as humans.

Is it really all we need? Or it is more that they threw a lot of hardware to it? What if if a part of its efficiency is because they threw a lot of GPUs with a huge network, rather than having a NN efficient by itself?

We see that: "AlphaGos Elo when it beat Fan Hui was 3140 using 1202 CPUs and 176 GPUs. Lee Sedol has an equivalent Elo to 3515 on the same scale (Elos on different scales aren't directly comparable). For each doubling of computer resources AlphaGo gains about 60 points of Elo."

It's a lot of hardware.

Re:AlphaGo beats the world champion Lee Sedol in first of fi... by javierluraschi on Wednesday March 09, 2016 @11:28AM (#11253854)
What does a lot of hardware means? The human brain has about 100B neurons with about 100+ dendrites per neuron, while AlphaGo has about 1K CPUs with about 2B transistors per CPU.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by nazka on Wednesday March 09, 2016 @01:25PM (#11254672)
I am wondering how much the amount of hardware they used had an effect on the bottom line, compared to the wisdom of their algorithm. Everybody knows it's a great step in AI. But how much? How much their algorithm is smart? Or simply put did they overfit by throwing a lot of layers and GPUs to the task? Or the algorithm is truly smart? What is the ratio of that.

It is the same question for the data they used. Facebook, Google and others seem to agree that, at the end, the quantity and quality of data are more important than the algorithm itself. So how much is it at play here? Knowing that will be able to show us why it is performing well and how much we can appreciate their work.

Re:AlphaGo beats the world champion Lee Sedol in first of fi... by javierluraschi on Wednesday March 09, 2016 @05:18PM (#11256111)
You can take a look at their paper... http://airesearch.com/wp-content/uploads/2016/01/deepmind-ma...

Basically, the ("lots of hardware") distributed implementation gets ~3100 points in the Elo rating against ~2900. ~2900 is still sufficient to win against Fan Hui. So I would say, that yes, this algorithm has most of the merit here.

Re:AlphaGo beats the world champion Lee Sedol in first of fi... by panic on Wednesday March 09, 2016 @03:33AM (#11251695)
It's totally impressive! I'm excited for the future of DeepMind and to see what other kinds of things we can build from neural networks. I'm just saying that we have to be careful with what kind of intelligence we ascribe to an algorithm like this. An AI is not going to take over the world by impressing us with its game-playing skills.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by JabavuAdams on Wednesday March 09, 2016 @07:54AM (#11252656)
> An AI is not going to take over the world by impressing us with its game-playing skills.

What if you train an AI to play an RTS where matter, energy, and time are the resources and the goal is to take over the world?

Re:AlphaGo beats the world champion Lee Sedol in first of fi... by argonaut on Wednesday March 09, 2016 @08:55AM (#11252936)
I'm not going to fear an AI whose idea of "the world" is a computer game, an AI that isn't even aware of the existence of the real world, and isn't even aware of the existence of the set of real world actions.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by PascalsMugger on Wednesday March 09, 2016 @10:14AM (#11253391)
I think, almost by definition, you won't be afraid of anything until it's already coming for you. By that time it will already have the capability to decide your future.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by argonaut on Wednesday March 09, 2016 @04:29PM (#11255865)
The thing is you are essentially making an unfalsifiable argument, invoking the existence of something that is merely imaginary.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by JabavuAdams on Wednesday March 09, 2016 @11:10AM (#11253744)
I should have put "RTS" in quotes. I'm referring to the real world -- the original RTS.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by nl on Wednesday March 09, 2016 @06:15AM (#11252245)
DeepMind is the company/lab Google bought. The algorithm is more properly called AlphaGo.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by taneq on Wednesday March 09, 2016 @05:14AM (#11252030)
> Forest for the trees.

Heh, nice one.

Re:AlphaGo beats the world champion Lee Sedol in first of fi... by Eliezer on Wednesday March 09, 2016 @03:11AM (#11251616)
I don't think it's accurate to call AlphaGo 'improved tree search' the way that Deep Blue was improved tree search. You could with equal justice call it an improved neural net.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by argonaut on Wednesday March 09, 2016 @03:16AM (#11251630)
The correct characterization is that it's a hybrid of deep learning and tree search. It's also a hybrid in the feature representation for the algorithm. There are raw features (board state), but there are also handcrafted features for the tree search.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by panic on Wednesday March 09, 2016 @03:20AM (#11251641)
I'd say both characterizations are accurate. Neither the tree search nor the neural net could have accomplished this on their own. But the essential interface to the algorithm is the tree search: it's picking the best move from a set of legal moves determined by formal game rules. The real world doesn't follow formal game rules. I find it difficult to see the progression from this victory to some kind of real-world AI takeover.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by kqr on Wednesday March 09, 2016 @06:30AM (#11252290)
Sure, but when you say "tree search" people think of the traditional kind of expert system-based tree search. Traditional tree search methods prune trees by approximating subtrees according to some very specific rules decided on by human experts. This means in a tree search system, the computer cannot actually evaluate a position any better than the humans that designed it. The way it performs better is by running very many of these evaluations deep down in the tree.

When you're talking about some other kind of evaluation function, such as Monte Carlo rollouts, you usually prefix that to the tree search (in the case of Monte Carlo rollouts, "Monte Carlo tree search" or MCTS) to indicate that besides the basic fundamental task common to almost all AIs (finding the optimal branches in a decision tree) it functions completely differently from the expert systems.

So is the case with this program, which (in a first pass) approximates subtrees by a trained neural net, rather than Monte Carlo rollouts or an expert system. So using terminology that suggests classical expert system tree search is bound to cause confusion (as you noticed).

Re:AlphaGo beats the world champion Lee Sedol in first of fi... by igravious on Wednesday March 09, 2016 @03:57AM (#11251764)
> The real world doesn't follow formal game rules.

Really?

Why not?

Re:AlphaGo beats the world champion Lee Sedol in first of fi... by argonaut on Wednesday March 09, 2016 @04:18AM (#11251847)
Infinite state/belief/world space. Infinite action space. It's not so much that there aren't rules (there are - physics), it's that the complexity of the full set of rules is exponential or super-exponential.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by ogrisel on Wednesday March 09, 2016 @04:45AM (#11251937)
You can do math with continuous and infinite dimensional spaces.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by argonaut on Wednesday March 09, 2016 @05:19AM (#11252052)
And? This does not address my argument that the complexity is beyond-combinatorially explosive (infinite spaces). I'm not talking about the space of possible board states. I'm talking about merely the set of all possible actions.

EDIT: clarified my language to address below reply.

Re:AlphaGo beats the world champion Lee Sedol in first of fi... by ogrisel on Wednesday March 09, 2016 @03:26PM (#11255509)
...and it's possible to train learning agents to sense and interact with a world described by high dimensional continuous vector spaces, for instance using conv nets (for sensing audio / video signals) and actor-critic to learn an continuous policy:

http://arxiv.org/abs/1509.02971

The fact that the (reinforcement) learning problem is hard or not is not directly related to whether the observation and action spaces are discrete or continuous.

Re:AlphaGo beats the world champion Lee Sedol in first of fi... by ogrisel on Wednesday March 09, 2016 @03:28PM (#11255514)
see also by the same team:

http://arxiv.org/abs/1602.01783

Re:AlphaGo beats the world champion Lee Sedol in first of fi... by argonaut on Wednesday March 09, 2016 @04:30PM (#11255872)
There is a near infinite number of such spaces.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by ogrisel on Thursday March 10, 2016 @03:57AM (#11258035)
I don't understand why would the "number of spaces" matters. What matters is can you design a learning algorithm that performs well in interesting spaces such as:

- discrete spaces such as atari games and go, - continuous spaces such as driving a car, controlling a robot or bid on a ad exchange.

Re:AlphaGo beats the world champion Lee Sedol in first of fi... by argonaut on Thursday March 10, 2016 @07:17AM (#11258721)
A really really large number of distinct decisions that need to be made. A car only needs to control a small set of actions (wheels, engine, a couple others I'm missing). A game player only needs to choose from a small set of actions (e.g. place piece at position X, move left/right/up/down/jump).
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by ogrisel on Thursday March 10, 2016 @08:34AM (#11259053)
A human brain also has a limited number of muscles to control to interact with the world.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by argonaut on Friday March 11, 2016 @07:57AM (#11266313)
And a much larger set of decisions.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by eru on Wednesday March 09, 2016 @06:19AM (#11252258)
Go is combinatorially explosive, too.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by vlowther on Wednesday March 09, 2016 @11:33AM (#11253905)
Your assertion that the universe has infinite anything is a common mistake. A stupidly large number of states is not infinite.
Re:AlphaGo beats the world champion Lee Sedol in first of fi... by zodiac on Wednesday March 09, 2016 @04:06AM (#11251808)
Well, by that logic, why can't you describe Lee Sedol as applying an "improved tree search" that happens to have really good move selection heuristics and static board evaluation function as well? After all, all good human Go players read ahead a few moves.