outsideline
Incorrect. It was named a Neural (Turing) machine for a reason. Maybe people should go back and dust off the white papers from the 70s like those who are borrowing from that era and respectfully giving credit where credit is due.
They do great work and they are making great progress in Artificial Intelligence. Many people are. Everything is a stepping stone. It serves no good to over-hype one person's stones over another's or ignore/downplay where they were inspired from. Notable visionaries of a past time were visionaries because they detailed the depths of their thinking and centered on the hows/whys. It seems it is fashionable now-a-days to do the exact opposite. This is to a disservice to learning and progress.
The whole point of the human brain is parallel processing. Extra-cellular chemical Gradients function the same way in the human brain and serve the same purposes. Take a look at the papers I linked.
> I don't know if they are copied or not from nature (doesnt look like).
Extra-cellular chemical Gradients. I linked to white papers that explain how memory is stored in them and shared across neurons. This is how it works in nature and biology.
They named their approach 'Synthetic Gradients'. An artificial form of the biological Gradient that is decoupled and lies outside of a neuron. They are clearly giving credit to nature.
They and many other people are improving mankind. Many others can improve mankind if there was less hype and more of a focus on where the ideas originated.
That was my point..
The behavior of people regarding selective 'hype' is one of the big reasons why a tremendous amount of deeply functional work that centers on hard intuitions and ideas for this area will remain closed source when a real break is made.
Enjoy the hype train I guess... They're operating on another level than anyone else.
Coming up with a functional systems architecture that ties the bits and pieces together is hard work. Understanding what is really happening in the human brain, how/why it is performing various functions, and how this provides for an intelligent architecture is hard work. Creating an 'aware' platform is hard and elusive work which is why people chase the low hanging fruit of optimization algorithms.
*Cheers
Making statements that allow people to see behind the curtains and maybe go off and make their own competitive models... Yes, this is a disservice to the advancement of A.I and should be downvoted : Removing the prestigious veil and illusion from published works.
NTMs memory is external in what sense? Please detail what this means in a 'functional' sense. It's biologically inspired. Neurons maintain memory beyond synaptic weights. The neuron models of present day A.I were basic. Someone comes along and sees the obvious : There is no computational model for how neurons utilize memory and suddenly they're thinking on another level? Give me a break..
Synthetic gradients have everything to do w/ electro-chemical gradients : http://www.nature.com/articles/srep14527 http://www.pnas.org/content/110/30/12456.full.pdf So, where is your establishment that I am incorrect. It is nowhere to be found. Again, biologically inspired computational models.
Oh look, someone published a paper back in June that is an implementation of Differentiable Neural Computers: https://arxiv.org/abs/1607.00036
It's hype and that is a disservice to the community of people completing similar work and taking similar approaches.
It would be an interesting discussion to have. That discussion was terminated in favor of downvoting me.
They're feeble to someone who isn't well informed on neuroscience. Thus, you'd rather be wow'd and believe in the fantasy that only a small segment of people can write computational models of biology.
Continue believing the hype. Rarely will someone be truthful and honest about where they got their ideas when hype follows. An interesting conversation could have transpired. Enjoy the feels from the downvotes.
Present day Neuron models lack an incredible number of functional features that are clearly present in the human brain.
NTMs = representing memory that is stored in neurons https://en.wikipedia.org/wiki/Neuronal_memory_allocation
Decoupled Neural Interfaces using Synthetic Gradients = https://en.wikipedia.org/wiki/Electrochemical_gradient
Differentiable Neural Computers = Won't specify what natural aspect of the brain this derives from.
Pick an aspect of a neuron or the brain that isn't modeled, write a model...
Bleeding edge + Operating on another level
The fact that someone is going out of there way to remove points from my posts so that this doesn't see tomorrow's foot traffic instead of replying and critiquing me just goes to show how truthful these statements are.
Anyone can create such models. No one has a monopoly or patent on how the brain functions. Thus, expect many models and approaches.. Some better than others.
You can down-vote all you want. The better model and architecture wins this game. It would help the community if people were honest about what's going on here but people instead want to believe in magic and subscribe to the idea that only a specific group of people are writing biologically inspired software and are capable authoring a model of what is clearly documented in the human brain. Interesting that this is the reception.
Here.. take the 'Axon Hillock' https://en.wikipedia.org/wiki/Axon_hillock code up a function for it, attach it to present day neuron models, make it do something fancy, write a white-paper and kazaam you're operating on another level..
Get it?
It's decoupled yet stores transient local information regarding previous neuron activity.
Another bio-inspired copy-pasta.
In the human brain, Neurons store an incredible amount of information. Neuron models in neural networks only did so with weights.
There is still a lack of understanding on how the human brain does it. Deep Mind grabbed a proven memory model from Alan Turing's work and applied it to the feature barren neuron models in use. Sprinkle magic ...
They are not operating on another level, they're bringing over features that are well documented in the human brain and in white papers from a past period when people actually thought deeply about this problem and applying it.
https://en.wikipedia.org/wiki/Bio-inspired_computing
There is no 'intuition' about the architecture. Study the human brain and copy pasta into the computing realm.
Others are doing this as well. If anyone bothered to read the white papers people publish, you'll see that many people have presented similar ideas over the years.
You can come up with your own neural Turing machine. Take a featureless neuron model, slap a memory module on it and you have a neural turing machine.
>A number of AI/ML startups would like to pursue AGI in the long-term, but why not make revenue along the way? That's exactly the dilemma that many capable researchers faced. Some chose to make money along the way. It shapes the way you then go about solving AGI down the road. It saps up your resources, imagination, and potential and it redirects it to shorter term thinking. Being in the space, there should be an understanding of Short-term vs. long-term thinking based on a reward system that targets short-term success.
> Still, there is at least one startup - Vicarious - focusing purely on long-term research towards AGI that VCs poured plenty of money into, and there are probably others. I'm familiar with Vicarious. You are correct and they notably publish far less details than others. They are funded by top names and are probably the least mentioned when people mention Ai. Why, given the impact that AGI can have, aren't more groups funded? A solution is seemingly right around the corner... Do people not see this?
Yes. Given the attitudes of people in the space and beyond and lack of funding why shouldn't the more valuable and fundamental research be kept secret? They're taking all of the risk to pursue something that everyone is saying is impossible. Why would they publish details? If more funding and support would come through maybe they would. Otherwise, they're seemingly developing a fundamental AGI that has the ability to operate under its own free-will and mechanisms. It is being said that it's impossible... Once it's completed no one will be able to deny it at that point. Maybe the money will come then... Maybe the openness will too.
> Your post implied that you have some special insight into research groups that are doing AGI research and are making steady progress. I have done plenty of Google searches and not run across any notable AGI progress.
Yes and I will not disclose where my insights derive from. I'm sure people have their ways of discovering it nonetheless. Some groups and individuals are developing conscious self-willed software solutions. I'm referencing them. Is that not what 'life' which intelligence springs from fundamentally is? As will be true for AGI... Not programs that mimic human behavior or optimization algorithms.. Software solutions that are truly aware.
Here's what you need to make an Ai Gym :
> Physics engine (pick your open source package)
> Simulator (pick your open source package)
> Write a thin wrapper around it
It can be done and has been done by many individuals in less than a week or two of dedicated work.
I mentioned George Hotz to highlight a capable individual with vision, passion, and capability who 'thinks different'. With many individuals, the capability is there. However, often times the vision, passion, or capability to think different isn't. Funny then that there becomes this hiring norm which attempts to find the 'most capable' individual and ignores the other more important characteristics.
Time will tell. I'm personally looking elsewhere beyond the names everyone mentions when it comes to development in this space.
Given that the public funding of basic research most always results in the ushering in of new technological paradigms that make corporations tens of billions if not hundreds of billions of dollars, more should be done to make sure they pay back into the system that ensures their constant success.
Thankfully we live in an age in which the most important research to developing fundamental things like Ai is 'time'. If you have an internet connection and a reasonably decent computer, you have enough resources to create AGi.
If you passionately care about seeing your idea through, save up money, move to a cheaper location, wire your computer to the internet, and get to work.
Once you've proven your vision, who can deny you then? Slam it on the boardroom table, disrupt present day business models, and be the next tech giant.
Most AGI focused ventures are not even in the U.S. A prominent one is in China. Another is in Europe. The ones in the U.S are privately funded beyond the big names. Furthermore, where is the U.S's equivalent of : https://www.humanbrainproject.eu/ ?
Where are the seasoned software engineers in the ranks at these highly funded silicon valley Ai companies? I typically see a roster of 'big names' and PhDs... Whereas, I look at the companies specifically pursuing AGI and I see a whole range of individuals.. PHD neuroscience, Senior game developer, Robotics Engineer, Senior firmware developer, Guy from down the hall who can code circles around silicon valley's most decorated engineer, etc etc.
Look at the Geohot story, that's the kind of individual and company who ushers in a new paradigm ....
If the solution requires you to 'think different', how do you expect to achieve it by padding your company with a bunch of individuals who are all centered on the same techniques from academia?
Don't stretch your mind to far on this one. If someone or various groups develop AGI and you have no stake in them, you will have no say in how it gets used. For once, people need to be honest with themselves about this.
You're not going to be able to dictate from a third party consortium of PhDs how someone should run their company or develop their software.
To be honest, most of the funding and deeply interesting work for ground breaking Ai research lies in the public and defense sector. They have more of an idea and vision for what Ai can be applied to than the private sector. Leaps and bounds beyond.
In the private sector, I get asked questions like 'Can it be used to predict if someone will click on an ad'. In the public sector, I get a 40 page technology acquisition pdf outlining a forward thinking application of Ai.
People forget that self driving cars arose from Darpa research. The latest innovations in computer security utilizing Ai came from a Darpa competition. IBM's neuromorphic chip : darpa funding. Darpa and other defense programs also have a slew of Ai development projects open to researchers.
Ultimately I find this a bit ridiculous given how vocal some individuals in the tech field are about the 'dangers of Ai'. They harp and harp about what could happen if more capable Ai gets developed but you don't see their money anywhere near the groups doing deeper development in Ai.
Lastly.. Yes, better solutions and more deeply inspired approaches to Ai will trump hoards of data and compute power any-day. Thank goodness for the public sector pushing forward fundamental research and development in Ai.
"At one point, his interviewer asked the question that is on all our minds: “Should wise people have known better?” Of course, they should have, Buffett replied, but there’s a “natural progression” to how good new ideas go wrong. He called this progression the “three I’s.” First come the innovators, who see opportunities that others don’t. Then come the imitators, who copy what the innovators have done. And then come the idiots, whose avarice undoes the very innovations they are trying to use to get rich."
Source : https://hbr.org/2008/10/wisdom-of-warren-buffet-on-imi
Cloud computing, big data, social media, and most of tech is in a prolonged idiot phase. A new paradigm will be ushered in and the cycle will begin again. Any more detail just highlights or attempts to justify the idiocy.
The cortical regions are good at creating hierarchical feature maps and we had a bunch of search algorithms laying around in the parts bin. Presto : Present day A.I.
This approach meshes well with 'big-data' companies in possession of large compute stacks and data-sets. So, its the direction things went.
A perfect opportunity for disruption.. The current wave is on borrowed time.
I was partly inspired to pursue the path I took in R&D by observing that the industry didn't seem to want to consider someone thinking different or working on the true foundation of A.I (the hard problems).
I figured, if I was able to write software up and down the stack for the billion dollar network infrastructure equipment that powers and services the internet, I probably knew what I was doing w.r.t to engineering.
A networked system with a missing foundation...The rest is history and I look forward to making disclosures about my work in the near future.
In the meantime, you should know that there are quite capable and industry proven individuals working on this. They aren't quacks, maintain graduate degrees from the top universities of America, and have a proven track record in the industry. The spotlight just doesn't shine in their direction. Of course, once a functional model is proof'd, I'm sure that will change. Such is the history of new paradigms and those who, through deep and new understanding, seek to usher them in...
So, the foundational problems remain...
They remain because there is no foundation to these cortical systems. Anyone who states this is railed and laughed at. So, you get what you get.....
The article states : "Machines that truly understand language would be incredibly useful–but we don’t know how to build them."
There are people and groups who know how to build them. They are focusing on the 'foundation' first. That is not where the spotlight or money are directed. So, they remain in the dark.
We gained head-winds with a very trivial model of neurons and cortex like hierarchical neural network designs and the money sent people off to the races. People began writing wrappers, stuccoing the top floor, hacking up scaffolding, applying any C.S concept they could find in the parts bin to fancify the top floor.
That's where all of the attention and money is.. What does your system do? What benchmark can it beat? What data can it classify? What cool trick can it do to impress us? So, you get impressive trick systems that require massive amounts of data, training, and answer maps to obscure the lack of intelligence. As there is none explicitly designed into these systems, the system cannot convey its understanding.
It's nothing more than an answer map w/ annealing routines and memory... Very similar to cortical regions.
The foundation and supporting layers up to the top have been ignored, aren't getting any spot-light or money, nor are the individuals who continue to toil on it.
They're considered to be 'philosophers' and jokers and not real scientist/engineers/industry leaders. The A.I space shuts out a huge pool of varying opinions via its : If you don't have a PHD, one need not apply. If you're approaching it from any other methods than the ones subscribed to and you're not a name, face, or have a laundry list of papers you get the : Good luck (thumbs up).
And people stand around and wonder why the fundamental problems remain? Come on...
In any event, it wont remain for long and that will be due to someone/groups actually investing the time and energy to build a sound foundation. This begins first and foremost by deep philosophical questions about the nature of the universe and intelligence. The answers derived serve as a guiding light for further along scientific and engineering pursuits.
This article should be : AI's lack of a foundation. Whose going to build it? Whose going to invest the time to understand what exactly it is as opposed to hacking away at it?
It's the truth but would be considered a 'hit piece'. Until someone constructs a proper foundation, no one is going to give credence to the idea that current A.I lacks it. Hindsight is 20-20 as is a force-fed neo-cortex.