News for hackers. Stuff that matters.
Re:Visualizing popular machine learning algorithms by indubitably on Saturday October 24, 2015 @06:48PM (#10445177)
Looks like K-nearest neighbor does pretty well.
Re:Visualizing popular machine learning algorithms by obmelvin on Saturday October 24, 2015 @07:17PM (#10445261)
Yes, k-nn is theoretically the one of the best ML algorithms in the sense that it will find the closest items in the training set. For classification or finding similar looking items it is great. However, it has pretty poor running times for evaluation of unseen data (http://nlp.stanford.edu/IR-book/html/htmledition/time-comple...). This is contrary to something like neural networks, which take a while to train, but then evaluate very quickly. For real world use the training times matter to an extent, but in a web app or real time application the latency from knn is just impractical.
Re:Visualizing popular machine learning algorithms by jsyedidia on Saturday October 24, 2015 @09:26PM (#10445574)
That's why we developed the "Boundary Forest" algorithm which is a fast nearest-neighbor type algorithm with generalization at least as good as K-NN, while being able to respond to queries very quickly.

It maintains trees of examples that let it train and respond to test queries in logarithmic time with the number of stored examples, which can be much less than the overall number of training samples. It thus maintains k-NN's property of very fast training time, and is also an online algorithm, and can be used for regression problems as well as classification.

See our paper that was presented at AAAI 2015 here: http://www.disneyresearch.com/publication/the-boundary-fores...

Re:Visualizing popular machine learning algorithms by wodenokoto on Sunday October 25, 2015 @12:26AM (#10445973)
It also suffers from the curse of dimensionality, making it weaker as the number of features increase.
Re:Visualizing popular machine learning algorithms by andreasvc on Sunday October 25, 2015 @05:46AM (#10446478)
I wouldn't call it theoretically the best. It is affected by outliers and doesn't make any generalization at training time. This latter point raises the questions whether it deserves the name learning. I would say linear models are typically a better learning algorithm; I wouldn't know what to call "the best" algorithm, but it might be deep learning nowadays.
Re:Visualizing popular machine learning algorithms by jules on Sunday October 25, 2015 @07:45AM (#10446678)
These visualisations are great but misleading regarding the performance of these classifiers. In practice you don't have a lot of data in a small number of dimensions (2 in this case). You have a little bit of data in zillions of dimensions. Think of classifying a 100x100 pixel image: that's 3x100x100=30000 dimensional data. You may not even have one training sample per class per dimension. Generalizing from comparatively little data to a very high dimensional space is the true difficulty of machine learning. Unfortunately you can't easily visualize that.
Re:Visualizing popular machine learning algorithms by darkmighty on Saturday October 24, 2015 @10:30PM (#10445720)
Try the "Island inside an island" test (put a blue cluster inside an orange island). Only k-means and SVM dealt with it satisfactorily.
Re:Visualizing popular machine learning algorithms by wodenokoto on Sunday October 25, 2015 @12:27AM (#10445975)
Shouldn't a neural net with sufficient unit do that too?
Re:Visualizing popular machine learning algorithms by neolefty on Sunday October 25, 2015 @06:49AM (#10446577)
Yes, if you increase the hidden layer from 5 to 10 nodes:

http://jsfiddle.net/udb95202/