Re:Machine learning for the impatient: algorithms tuning alg...
by danger on Monday July 23, 2012 @02:41PM
(#4282404)
Also, for people who are interested in the application of predicting college basketball with machine learning, there's a Google group that is worth joining:
Re:Machine learning for the impatient: algorithms tuning alg...
by aelaguiz on Monday July 23, 2012 @02:48PM
(#4282432)
Thanks for the information! I've updated the article to reflect this.
Here's a question: where does "the field" hang out? Is there a cohesive community of any sort?
Re:Machine learning for the impatient: algorithms tuning alg...
by danger on Monday July 23, 2012 @02:51PM
(#4282448)
I'd say the closest thing to a cohesive community would be the MetaOptimize Q&A forum, but maybe others have other suggestions:
Re:Machine learning for the impatient: algorithms tuning alg...
by gammarator on Tuesday July 24, 2012 @12:23AM
(#4284340)
The Kaggle forums can be a good resource, and competing there is a good way to polish your skills.
Also, you might check out Random Forest algorithms--they're high performance but still very beginner friendly, as there aren't many parameters to tune. There's a nice implementation in the excellent scikits-learn python library.
Re:Machine learning for the impatient: algorithms tuning alg...
by pjin on Monday July 23, 2012 @03:01PM
(#4282504)
Hmm, so gaussian distributions are easy to use and ubiquitous and all (they're the basis functions used in SVM), except that I don't see any reason for them to be priors here. But since 2012 > 2008 I feel like I'm obligated (and I'm semi-trolling) to point out the obvious about lazy assumptions based on "flexibility and tractability", which is that they can implode hilariously in your face. C.f. the financial crisis.
[1] http://econometricsense.blogspot.com/2011/03/copula-function...
Re:Machine learning for the impatient: algorithms tuning alg...
by bayesregressor on Monday July 23, 2012 @03:23PM
(#4282628)
I think you're confusing the Gaussian "process" used in Bayesian optimization with a standard Gaussian distribution. They are very different things - as are Gaussian copulas and what is referred to as the 'Gaussian kernel' (which is not actually a distribution at all) in the SVM. The Gaussian process is a distribution over functions, the properties of which are governed by the covariance function - so the prior over the function, or the assumption about its complexity and form, is determined by the choice of covariance function. Of course it is very important to choose a prior that corresponds to the functional form you are interested in, which is actually discussed and empirically validated in the literature referred to in that post. It's a bit ironic that you are claiming to point out the dangers of making lazy assumptions by doing exactly that.
The basic idea of automatically tuning hyperparameters (the things this post discusses tuning with genetic algorithms) is cool, though, and is becoming a popular subject in machine learning research. A couple recent research papers on the topic are pretty readable:
Algorithms for Hyper-Parameter Optimization:
http://books.nips.cc/papers/files/nips24/NIPS2011_1385.pdf
Practical Bayesian Optimization of Machine Learning Algorithms:
http://arxiv.org/abs/1206.2944