News for hackers. Stuff that matters.
Re:The Google Brain Team – Looking Back on 2016 by ilaksh on Thursday January 12, 2017 @05:03PM (#13386563)
Personally I think that within X years (could be only 10, or 5, or probably less) we will see human-level AGI. This is because the grounded (based on granular sense/motor) deep learning systems such as Deep Mind are being applied in varied virtual environments and some with gradual development of skills and knowledge. This type of approach will enable truly general agents.

So we better hope that the BCIs develop quickly as well.

Re:The Google Brain Team – Looking Back on 2016 by fat-chunk on Thursday January 12, 2017 @05:46PM (#13386873)
I work in AI, specifically in the field of deep learning applied to computer vision. Not trying to discredit your comment, but can you provide some sources indicating that the type of models being trained which you have mentioned are approaching the notion of a general agent?

I haven't read much of the literature around deep learning, mostly only what has been applied to computer vision. But from what I understand, the general consensus is that the current crop of state-of-the-art deep learning models are very good at performing specific tasks (machine translation, object recognition in images, etc.), but are not so good at generalising across multiple fields. This seems more relevant to the domain of reinforcement learning (which does indeed include deep learning), which has proven to be a very difficult problem to solve.

Re:The Google Brain Team – Looking Back on 2016 by ilaksh on Friday January 13, 2017 @01:29AM (#13388870)
Having an agent with sensory and motor terminals in diverse contexts gradually trained on increasingly complex knowledge and tasks is key I believe. Like I mentioned see Deep Mind's work. Also see field of AGI which does exist (for example search 'AGI-16 intelligence' on youtube).

I am working on a webpage to try to break down why I think this is coming so fast.

https://www.youtube.com/watch?v=BP7vhBaBDyk&t=4589s General Reinforcement Learning

https://www.youtube.com/watch?v=T9eSVYLSSrs The Emotional Mechanisms in NARS

https://www.youtube.com/watch?v=eVrflIw6sGg&t=4056s AGI-15 Keynote by Jürgen Schmidhuber - The Deep Learning RNNaissance

Re:The Google Brain Team – Looking Back on 2016 by argonaut on Friday January 13, 2017 @02:24AM (#13389068)
Don't read too much into titles.

AGI(-16) is a cognitive science / philosophy conference (and not particularly high impact). Similarly, NIPS is not really about neural information processing.

Re:The Google Brain Team – Looking Back on 2016 by ilaksh on Friday January 13, 2017 @05:05AM (#13389708)
I'm not reading into titles. I watched a lot of the videos. Don't dismiss it on your superficial evaluation of titles or prejudice about the conference.

AGI is in fact a developed field with key insights into general intelligence. You should study it.

Re:The Google Brain Team – Looking Back on 2016 by nl on Friday January 13, 2017 @06:59AM (#13390137)
Argonaut works in the field. I'm pretty sure (s)he knows what NIPS is and has thought about AGI some too.
Re:The Google Brain Team – Looking Back on 2016 by argonaut on Friday January 13, 2017 @09:40AM (#13391079)
I comment a lot on machine learning but I don't actually work in the field actively. I did some research in college.

There aren't many active researchers/experts commenting on HN (better things to do), which is IMO a big issue with the ML-related discussion quality on HN (it's basically 90% futurism/speculation).

Re:The Google Brain Team – Looking Back on 2016 by Florin_Andrei on Thursday January 12, 2017 @08:36PM (#13387743)
The current AIs are all "base level". There's no "meta" level processing there.

The human brain is full of meta upon meta levels. Networks on top of other networks, and so on. Abstractization. Correlation between different domains. Feedback loops. There will be no AGI until we start building that kind of architectures.

We're currently only making individual Lego bricks. We haven't started to assemble them yet.

Re:The Google Brain Team – Looking Back on 2016 by ilaksh on Friday January 13, 2017 @01:24AM (#13388857)
See the field of AGI.

Networks on top of other networks -- about any recursive neural network could be in that category. Also there are multiple projects exploring metalearning on deep NNs where it learns the network topology, i.e. learning how to learn.

Abstraction in single domains is very commonly achieved with various types of hierarchy in NN and non-NN systems. Correlation between different domains has fewer examples simply because most AI systems do not deal with multiple domains, but they do exist.

Feedback loops are common and fundamental to AI and simpler non-AI control systems.

Re:The Google Brain Team – Looking Back on 2016 by lend000 on Thursday January 12, 2017 @05:27PM (#13386746)
> (could be only 10, or 5, or probably less)

I don't know when, but I can guarantee it won't be within 5 years. The reason: digital hardware improvements are slowing down. Today's best hardware can only run a model with the number of nodes in a human brain at 1/2400th speed, and probably with an oversimplified model: http://www.telegraph.co.uk/technology/10567942/Supercomputer...

I don't think we will reach human level AI on digital hardware; it just doesn't really make sense with the Von Neumann bottleneck. Parallel, analog computation is the way forward and that has yet to be commercialized, so we probably have a ways to go.

That being said, AI is getting very impressive on the software side, and with the use of TPU's, GPU's, and FPGA's, we can definitely look forward to compelling use cases where these algorithms have apparently creative intelligence, but only within an extremely specific category (when compared to the generalities of biological analog brains).

Re:The Google Brain Team – Looking Back on 2016 by lend000 on Thursday January 12, 2017 @06:04PM (#13386978)
I'm being down-voted to oblivion by all the pure software engineers on here who think their trade is being challenged -- well, it is. The fact of the matter is that the world runs on hardware and analog hardware is orders of magnitude more efficient.

It's definitely worth every software engineer's time to learn how computers work from all the way down at the laws of electricity, basic signal processing/analog circuits, up to semiconductor physics and microarchitecture.

Re:The Google Brain Team – Looking Back on 2016 by kevinnk on Thursday January 12, 2017 @07:19PM (#13387403)
I'm a hardware engineer and I downvoted you because your comment is shallow and uninformative. The "Von Neumann bottleneck" has nothing to do with digital vs analog hardware (rather it's about the separation between storage and computation) and the idea that analog computation is the way forward requires a much longer argument before it's even surface level plausible.
Re:The Google Brain Team – Looking Back on 2016 by lend000 on Thursday January 12, 2017 @07:29PM (#13387448)
Unfortunately, analog computation is not considered an important addition to the electrical engineering curriculum, even in computer architecture concentrations. I'll gladly explain what I know if you're genuinely interested; in general, you have to hope that people have some context when you make a comment in HN, but there are very few analog-focused people here. But anyway, that's where DARPA is putting their money: http://www.artificialbrains.com/darpa-synapse-program

I definitely see how my language would convey that whereas I mean to refer to digital computers, not all hardware, when I mentioned the Von Neumann bottleneck. There are different arguments against digital hardware in general, even though FPGA's can be about an order of magnitude more efficient than microprocessors. Modern analog goes much further, although we're still not very good at 'programming' it, which is why I think AGI is a ways out.

Re:The Google Brain Team – Looking Back on 2016 by kevinnk on Thursday January 12, 2017 @08:46PM (#13387784)
The main problem with analog computation is lack of reliability (analog is not always deterministic and manufacturing defects are more difficult to eliminate) and density (analog components are usually much larger, in part to increase reliability). That's a huge blocker for faster chips. How do you think we will get around those problems?

Also, perhaps I'm misreading their paper[1], but that DARPA project is still using digital logic. In fact, they specifically say that they decided against using analog computation due to the increased density of digitial (one of the main problems with analog computing, as I mentioned). Regardless, it's pretty clear that the project doesn't have the goal of replacing general purpose computing, so I'm curious why you consider this evidence that analog computing will replace digital computing?

Finally, in what way are FPGAs much closer to analog computing than microprocessors? The advantages of FPGAs (configurability) are orthogonal to analog vs digital.

[1] http://www.modha.org/papers/013.CICC2.pdf

Re:The Google Brain Team – Looking Back on 2016 by lend000 on Thursday January 12, 2017 @10:11PM (#13388117)
Who said anything about replacing digital? You are clearly more intelligent than my calculator, and yet I could program my calculator to do computations that would be practically impossible for you (unless you happen to be a savant). Strong AI will rely on analog, but through future integration with digital, it can achieve the best of both worlds.

The topic is AI, where efficient, large scale neural nets will require fundamentally different hardware, like the programmable mixed signal Samsung boards used by Synapse. Those are best described as specialized FPAA's and they certainly do rely on digital for programming, and special (although still solid state circuits) for pseudo-analog properties. However, the most interesting parts are yet to come, and are being developed by partners/founders of the program: http://knowm.org/about/

The next generation boards are likely to contain memristors and self organizing memory, with a structure somewhere between the organization level of FPGA LUT blocks and biological nervous systems. This will obviously need to interface with/through digital circuits to be of any use.

Re:The Google Brain Team – Looking Back on 2016 by kevinnk on Thursday January 12, 2017 @11:21PM (#13388363)
The way I read your second post was that analog computing is more efficient in general; it sounds like you don't actually believe that. Fair enough.

However, I still take issue with your overall point.

> Efficient, large scale neural nets will require fundamentally different hardware, like the programmable mixed signal Samsung boards used by Synapse.

I think it's at least plausible that we'll see gains from fundamentally different architectures (namely combining memory and computation) but where's the evidence that we need to switch from digital to analog? Where's the evidence that analog is even better for an architecture like that? Again, the DARPA project you linked actually gave evidence of the opposite.

> The next generation boards are likely to contain memristors and self organizing memory, with a structure somewhere between the organization level of FPGA LUT blocks and biological nervous systems.

You keep talking as if there's some obvious reason that analog is more efficient/faster/better for the types of architectures you're talking about. I'm not seeing that reason.

Re:The Google Brain Team – Looking Back on 2016 by lend000 on Friday January 13, 2017 @12:05AM (#13388530)
Here's a better attempt to explain it: we are using digital circuits to basically simulate analog circuits with analog properties. It should be intuitive that analog is more efficient for representing analog abstractions (neural networks, which inherently copy analog structures). Digital hardware adds an additional abstraction layer built out of components that do not behave like the desired structures (neurons) and are basically running simulations. It's no surprise that some of the most successful models have shifted away from the perceptron-style representation and use more mathematical models to better leverage digital hardware for the node simulations, but they still rely on the basic models of weights and feedback cycles which require clock cycles and controlled computation on digital circuits.

Digital is more efficient for some problems in the sense that it's the only way to achieve precise computational results, but even when it comes to mathematical arithmetic, the circuitry to add two analog signals is much faster and simpler (two wires and a resistor), as is the design of a multiplier (one transistor).

It's definitely a complex subject, but I don't see why you would find it so difficult to believe -- the most intelligent machines we know of are analog and decentralized (like human brains).

Re:The Google Brain Team – Looking Back on 2016 by kevinnk on Tuesday January 17, 2017 @04:44AM (#13416195)
> It should be intuitive that analog is more efficient for representing analog abstractions (neural networks, which inherently copy analog structures).

That's not necessary true; again, see the DARPA project you listed.

> Digital hardware adds an additional abstraction layer built out of components that do not behave like the desired structures (neurons) and are basically running simulations.

The analog hardware doesn't necessarily behave like the desired components either (for example, due to manufacturing errors). With digital you get to choose your accuracy and with much greater control.

> Digital is more efficient for some problems in the sense that it's the only way to achieve precise computational results, but even when it comes to mathematical arithmetic, the circuitry to add two analog signals is much faster and simpler (two wires and a resistor), as is the design of a multiplier (one transistor).

Unless you don't care about noise at all, your design will be much more complicated than a single transistor. Either way, it's actually much easier to do this sort of "approximate computing" in the digital domain.

> It's definitely a complex subject, but I don't see why you would find it so difficult to believe -- the most intelligent machines we know of are analog and decentralized (like human brains).

A bird has feathers yet our most efficient airplanes don't use them; there's no reason at all to believe that just because the brain is analog, our design should be as well.

Re:The Google Brain Team – Looking Back on 2016 by fooker on Thursday January 12, 2017 @05:11PM (#13386621)
Or there will be another AI winter. There is no way to tell now.
Re:The Google Brain Team – Looking Back on 2016 by nl on Friday January 13, 2017 @07:01AM (#13390150)
There's not going to be another AI winter. I'm sure it will drop down the hype curve, but it's producing too much value already.
Re:The Google Brain Team – Looking Back on 2016 by fooker on Friday January 13, 2017 @07:58PM (#13396146)
The traditional search algorithms and heuristics from last time also did so, leading up to practical search engines and social networks.
Re:The Google Brain Team – Looking Back on 2016 by maverick_iceman on Thursday January 12, 2017 @05:38PM (#13386824)
The singularity is far.[1]

[1] http://www.scottaaronson.com/blog/?p=346