Re:Deep Learning With Commodity Off-The-Shelf High Performan...
by dumitrue on Tuesday June 18, 2013 @01:58AM
(#5897805)
It's possible that the underlying model is just not particularly good at learning from data. 11B parameters is a lot of free parameters to learn -- for instance, the main competitor to that paradigm is the work by Krizhevsky et al., which are convolutional networks with lots of parameter sharing, and I think they get better performance (on a comparable task) with ~60M free parameters.
This is becoming accessible for everyone. It's both exciting and terrifying. It has the potential to save humanity from itself or condemn us to totalitarianism. I am sure machine learning and NLP and statistical models are what enables them to analyze the data they collect on us.
Big, fast noSQL tables, clustering technology (map-reduce) and machine learning are what allows these guys to do what they do. Our most prized toys became our enemies.