News for hackers. Stuff that matters.
Re:Deep Learning With Commodity Off-The-Shelf High Performan... by visarga on Tuesday June 18, 2013 @12:23AM (#5897532)
Weird that a team led by Andrew Ng couldn't do better with the 11 billion parameters model than with the 1 billion one.

This is becoming accessible for everyone. It's both exciting and terrifying. It has the potential to save humanity from itself or condemn us to totalitarianism. I am sure machine learning and NLP and statistical models are what enables them to analyze the data they collect on us.

Big, fast noSQL tables, clustering technology (map-reduce) and machine learning are what allows these guys to do what they do. Our most prized toys became our enemies.

Re:Deep Learning With Commodity Off-The-Shelf High Performan... by dumitrue on Tuesday June 18, 2013 @01:58AM (#5897805)
It's possible that the underlying model is just not particularly good at learning from data. 11B parameters is a lot of free parameters to learn -- for instance, the main competitor to that paradigm is the work by Krizhevsky et al., which are convolutional networks with lots of parameter sharing, and I think they get better performance (on a comparable task) with ~60M free parameters.