News for hackers. Stuff that matters.
Re:A Non-Mathematical Introduction to Using Neural Networks by jules on Wednesday June 09, 2010 @02:18PM (#1417777)
The mathematical explanation is surprisingly simple. A 2 layer neural network consists of 2 matrices A and B and an activation function f. The neural network computes this function where in and out are vectors:

    out = f(A*f(B*in))
where f is applied to each element of a vector. You can see how this generalizes to more layers.

Neural network learning algorithms are given (in,out) pairs and try to find A and B to minimize the mean square error, for example by using stochastic gradient descent.

Re:A Non-Mathematical Introduction to Using Neural Networks by sqrt17 on Wednesday June 09, 2010 @05:15PM (#1418314)
This is a good summary. Now, here's what you need to know to have it work:

* initialize weights to small random values (otherwise it will quickly get stuck in a non-desirable local optimum)

* you need to decrease the learning rate of the stochastic descent according to a 1/(n+n0) or similar schedule, otherwise there's a risk of the whole thing swinging back and forth

* it often helps to normalize the input values so that they have a standard deviation of 1 and an expectation of zero. Or at least it's not good to have input values differ too much in scale.

* if you're training something that should give yes/no answers, you don't really want to minimize the mean square error, but a logistic loss.

Some of these parts are usually built into your NN toolkit, so you don't have to worry about them. Or they're not (e.g., choosing a good learning rate), in which case you're just screwed if you don't know what happens and why.

But you're right, mathematically ANNs are quite simple (and, as one has found out, don't really do similar things to what actual neurons do).

Re:A Non-Mathematical Introduction to Using Neural Networks by gtt on Wednesday June 09, 2010 @06:59PM (#1418602)
Could you provide a link to mathematical sound introduction into neural networks?

I was always turned out because it was always explained in some magical manner...

Re:A Non-Mathematical Introduction to Using Neural Networks by dododo on Wednesday June 09, 2010 @07:20PM (#1418649)
if f is the logistic function, then neural networks basically correspond to logistic regression.

http://en.wikipedia.org/wiki/Logistic_regression

which has been around since the 1940s... (multilayer neural networks correspond to hierarchical logistic regression--just plug them together). this should be in any reasonable stats book.

if you mean mathematically sound in terms of learning, etc, then chris bishops "neural networks and pattern recognition" is pretty good, full of sage advice and justification.