News for hackers. Stuff that matters.
Re:A Basic Introduction to Neural Networks (2007) by annnnd on Wednesday December 14, 2016 @02:18AM (#13174399)
Looking at headers:

    Last-Modified: Tue, 30 Apr 1996 18:53:31 GMT
Still, looks good, if a bit dated. For me, these two tutorials were the ones that helped me understand NN the most:

- http://neuralnetworksanddeeplearning.com/

- https://mattmazur.com/2015/03/17/a-step-by-step-backpropagat...

Re:A Basic Introduction to Neural Networks (2007) by whym on Wednesday December 14, 2016 @02:56AM (#13174515)
It's refreshing to see a page with Last-Modified actually showing the date the document was last modified. Nowadays, too many websites report the current date even when no single byte of the content is changed since publication.
Re:A Basic Introduction to Neural Networks (2007) by hashin on Wednesday December 14, 2016 @04:51AM (#13174891)
I have completed this tutorial by Micheal Nielsen and it has enormously supplemented my understanding of the subject. The good part is that it just had enough mathematical rigour the subject demands.

I didn't get lost in the details and was able to see the bigger picture after going through his work. It has brought in enormous change to my career and life. I can't thank him enough.

Re:A Basic Introduction to Neural Networks (2007) by nwj on Wednesday December 14, 2016 @08:28AM (#13175753)
Just adding a +1 for http://neuralnetworksanddeeplearning.com/. It is really excellent. Clear, but mathematically rigorous. Highly recommended.
Re:A Basic Introduction to Neural Networks (2007) by annnnd on Thursday December 15, 2016 @02:22AM (#13183039)
Agreed - it gives a very solid overview of the field. Truth be told, it was a bit too mathematically rigorous for my taste, so I had trouble producing code for backpropagation. However with the second link (backpropagation step-by-step, with numbers so you can check progress) and then TensorFlow Jupyter notebooks I think I cracked it. Now I just an excuse to use ANN at work... ;)
Re:A Basic Introduction to Neural Networks (2007) by eriknstr on Wednesday December 14, 2016 @01:48PM (#13178554)
Wonder where OP got the idea that this was from 2007.
Re:A Basic Introduction to Neural Networks (2007) by argonaut on Wednesday December 14, 2016 @01:30AM (#13174272)
This is neither basic nor an introduction, and has several inaccuracies. My guess is the info is from the 90s era of neural networks.

Reply to below: it uses pretty complicated language and doesn't explain many terms (e.g. activation functions, supervised learning, etc. are not explained). Errors: the statements the article makes about the error surface are really misleading and in some places wrong. Same for the statement about sigmoids. Some of their definitions are also wrong today, such as "epoch." The list goes on.

Re:A Basic Introduction to Neural Networks (2007) by Karlozkiller on Wednesday December 14, 2016 @02:14AM (#13174392)
What do you think is wrong or outdated with the article? It's not that far off how ANN are described in the beginning of current scientific articles. Backpropagation is definitely still relevant even though changes can be made to it and the networks have grown in size and changed in layout. The basics are more or less the same and from my quick scan they seem accurately described here. What is it you expect from a simple intro?
Re:A Basic Introduction to Neural Networks (2007) by zardeh on Wednesday December 14, 2016 @02:47AM (#13174488)
While I don't share his total disdain for the article, he got the date spot on, and I'd say that

- no mention of non-sigmoid activation functions - calling overfitting 'grandmothering' and - diving comparatively deep into gradient descent (which while absolutely the most common, is not the only way to train a neural network), while being an inch deep on practically every other aspect.

Its a bit of a confused intro that simultaneously tries to teach you the basics while throwing jargon at you ('delta-rule', 'beta-coefficient', 'hyperparaboloid', etc.).

Re:A Basic Introduction to Neural Networks (2007) by amelius on Wednesday December 14, 2016 @11:42AM (#13177320)
What actually is new since the 90s, besides convnets?

Is ML becoming more prevalent because of theoretical breakthroughs, or is it because of hardware improvements, or perhaps because there is more training data available now?

Re:A Basic Introduction to Neural Networks (2007) by thegeomaster on Wednesday December 14, 2016 @01:32PM (#13178383)
What enabled the ML revolution we're witnessing now is mostly the advent of powerful and low-cost GPGPUs and the abundance of training data, as you mentioned. Of course, there are theoretical breakthroughs and interesting new applications [1], but it wasn't the deciding factor. Even ConvNets aren't new. For example, AlexNet is basically a much larger reimplementation of the original Lenet idea [2] from 1998.

This is not to discredit any new work that's being done. I think it's awesome to see so much progress in a field that's certainly on its way of defining an era. I'm just pointing out that we have known about most of the fundamentals for a long time.

[1]: https://tryolabs.com/blog/2016/12/06/major-advancements-deep...

[2]: http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf

Re:A Basic Introduction to Neural Networks (2007) by argonaut on Friday December 16, 2016 @02:50PM (#13196173)
It's not about newness. There are several things that are flat out wrong in the article.
Re:A Basic Introduction to Neural Networks (2007) by ge96 on Wednesday December 14, 2016 @06:40AM (#13175279)
I have to do some research. I'm primarily commenting on this to keep track of it.

Ultimately I want technology to be my better half. Fill in my gaps if you will. I want my vitals linked to a server, a "CRON" job monitoring my sleep/wake cycles. I've been doing some scraping. I'd like to develop my own thing runs "autonomously and grows"

Gotta read, between this and machine learning. Gotta focus though. Not sure when I'll come back to this. Thanks for posting this.

Re:A Basic Introduction to Neural Networks (2007) by ge96 on Wednesday December 14, 2016 @06:53AM (#13175326)
After reading that article and what I just described, it does not seem like neural networks is what I'm after. Still it is something to use.

I wanted to write a "quick search" scrape pages related to a search (though last time I checked, the Google API wasn't available anymore where you could search Google back end.)

Anyway. Maybe a source like Wikipedia. Still parsing words and assigning them values...

That part about using ANNs for finding regularities in patterns. That sounds interesting. It seems many successes came by determining the next step in some form of evolution whether it was a product or service. Blockbuster to Netflix, Blackberry's to iPhones, etc... Maybe look at something like that.

At any rate, gotta read. That was quite informative. The whole thing of "not knowing what it actually does" is pretty nuts too. Give it data and watch it go!

Re:A Basic Introduction to Neural Networks (2007) by rawnlq on Wednesday December 14, 2016 @02:11AM (#13174380)
Better off learning from the man himself: https://www.coursera.org/learn/neural-networks
Re:A Basic Introduction to Neural Networks (2007) by derekmcloughlin on Wednesday December 14, 2016 @06:38AM (#13175273)
For a good basic intro, this is excellent:

https://www.amazon.com/Make-Your-Own-Neural-Network-ebook/dp...

Re:A Basic Introduction to Neural Networks (2007) by vonnik on Wednesday December 14, 2016 @09:31AM (#13176156)
We wrote a decent intro to neural nets here: https://deeplearning4j.org/neuralnet-overview