One model to learn them all

Re:One model to learn them all by YeGoblynQueenne on Saturday January 13, 2018 @08:20AM (#16139498)

For me, as a non-practitioner (I'm happy with my GOFAI, thank you), the big problem with neural nets is that there are too many architectures, each tailored to a specific problem. On the one hand it's great that there's a broad toolset, on the other hand there are so many competing claims about best-of-class performance that it's hard to know what is even the state of the art. There is too much noise, you know?

So it'd be nice to see a result that reduced the noise a bit. I'm afraid this one doesn't fit the bill. It's not so much reducing architectural options, as piling even more architecture on top of the already sprawling mass of architecture. It's got architecture hanging from its architecture!

I mean, come on- one component is a gated-mixture-of-experts, which is to say, a collective of feed-forward nets. There are just so many layers upon layers of choices to make for each type of network to use for each component of the entire model. How do you make these choices?

Or, think about how long this specific architecture is going to give state-of-the art results (as claimed). It uses normalised ReLu convolution blocks- the height of fashion, at the moment, but what happens in four years from now, when nobody uses that anymore, because it's so 2010's?

ANN research frustrates me like this. It's describing an art form, but I'm not sure that's really the most useful thing to do.

[ Parent | on HN ]

Re:One model to learn them all by npatrick04 on Saturday January 13, 2018 @01:27PM (#16140558)

I get your frustration with architecture of architecture. However this kind of AOA has been going on for a long time in a similar domain of control theory.

Model Predictive Control is a method of applying a model of a process to estimate it's state. Per Wikipedia, it's been used since the 80s. When you apply it to tracking something you aren't in control of, say an enemy plane, you need to use multiple models, and then pick the most probable solution.

It's not elegant, but it's not actually a bad way to get good results.

[ Parent | on HN ]

Re:One model to learn them all by michaelmior on Saturday January 13, 2018 @08:37AM (#16139538)

I understand the frustration. I'm just getting into ANNs myself and the options are overwhelming. Trying to keep up with the state of the art is definitely a challenge. However, if you have a model that gives you the accuracy you need for whatever problem you're trying to solve, why not just stick with it? One of the nice things about using Neural nets these days is there's a lot of great software and infrastructure available for training and serving models.

[ Parent | on HN ]

Re:One model to learn them all by ghthor on Saturday January 13, 2018 @11:45AM (#16140164)

You should look at the models being produced by Jeff Hawkins and numenta. Their neuron is based on the neuron of our neocortex which are homogeneous. Theyve had great success using their ANN across different domains with very little to no differences in settings. The magic really is in the encoding of the information stream into the Sparse Ditributed Representation that best represents the features you're looking for.

[ Parent | on HN ]