News for hackers. Stuff that matters.
Re:A Tour of Machine Learning Algorithms by nostrademons on Thursday May 22, 2014 @01:56PM (#7785183)
This table of contents is beginning to seem very familiar (I've done Google's internal machine-learning course and looked at several other courses/tutorials on the net), but what I'm wondering is: how do you do the more basic operations of feature and parameter selection? When I've tried using machine learning in my daily work, I usually top out at "not quite good enough", largely because the features I end up choosing have a lot of noise in their data. Meanwhile, more experienced experts (those who have been doing this for 10+ years) produce much better results in much shorter time periods because they use totally off-the-wall features that I would never have considered. When they've presented at these classes, they've said that feature selection is really the vast majority of the work in practical machine learning. How do you develop an intuition about which features will be useful? Are there any resources that are more like "A practitioner's guide to machine learning", talking about the stuff that's useful in practice and not just in theory?
Re:A Tour of Machine Learning Algorithms by eshvk on Thursday May 22, 2014 @03:58PM (#7786078)
> This table of contents is beginning to seem very familiar (I've done Google's internal machine-learning course and looked at several other courses/tutorials on the net), but what I'm wondering is: how do you do the more basic operations of feature and parameter selection?

I have thought about this problem for a long time. Initially, I used to think it was a mere matter of finding clever Machine Learning algorithms that could auto-magically detect features for you. There are others who think it is a matter of throwing a bunch of subject matter experts into the problem and building feature vectors. I have worked with both philosophies of people. I think the answer lies somewhere in the middle. And it is a fucking hard problem.

You need to do some clever algorithmic work. Maybe use a PCA or your favorite dimensionality reduction trick but also be clever enough (mathematically that is) to realize when PCA is brittle and when not to use it. You also need to have enough understanding of the subject to be able to ask the right questions from your subject matter experts. It is easy to get five people in a room and ask them to describe a song. But it is much harder to find out the right way of asking them: Is this song the kind of song that you would dance to while drunk on three beers in a podunk town in the Mid West? That requires experience to get that sort of intuition.

The only thing I can recommend (pun unintended) is to keep solving these problems, focus on the domain(s) that you are deeply passionate about. Don't spread yourself too thin and make yourself to be a general purpose Machine Learner. That is the only way to get to building beautiful end to end products.

Re:A Tour of Machine Learning Algorithms by gms on Thursday May 22, 2014 @06:20PM (#7786911)
Experience, as there is no canonical feature set that applies to all problems. Feature sets instead tend to be domain-specific.

The usual iteration loop for a machine-learning system is:

1) Try features.

2) Evaluate your system. If you are happy with your scores/metrics, stop.

3) Do error analysis/run diagnostics. Go to 1.

It is the last step I find inexperienced people usually lacking. You need to examine your errors and find commonalities among them.

For example, is there a certain class of error that hints at a certain feature you haven't tried? Do your existing features provide decent results to begin with? Or maybe you should just get more data? Whatever path you take, it needs to be defensible.

So, in terms of general advice:

1) Realise it's an iteration loop, not a fixed recipe.

2) Be able to justify why you are trying whatever it is you are trying.

With time, you'll develop intuition for given domains.

Re:A Tour of Machine Learning Algorithms by jimbokun on Friday May 23, 2014 @08:40AM (#7789048)
"2) Be able to justify why you are trying whatever it is you are trying."

This is just a restatement, I think, of the question you are responding to. I suspect a book on this topic would be quite popular.

Re:A Tour of Machine Learning Algorithms by elliott34 on Friday May 23, 2014 @02:39PM (#7790700)
So it's not just me! The stuff people do in Kaggle competitions is mind blowing. They do models within models within models, do weird interaction terms with their features and transform them with different functions, although most of that is to go from 94% accuracy to 97% accuracy...
Re:A Tour of Machine Learning Algorithms by nanidin on Thursday May 22, 2014 @02:01PM (#7785213)
Unless you have a massive amount of features already, can't you just try throwing stuff at the wall until something sticks? And even then you could run PCA on your features to reduce dimensionality.